DZone.com Feed

DZone.com Feed

Robot Framework With Selenium and Python: All You Need to Know

Posted: 17 Nov 2020 02:59 PM PST

With 5000+ stars and 1500+ forks on GitHub, the Robot framework has been a go-to option for many organizations who are aiming for Agile and Test Driven Development (TDD) where developers should write functional code only when there is a test that has failed. The robot framework allows acceptance testing, behavior-driven testing, Robotic Process Automation (RPA), and Acceptance test-driven development (ATDD).

It offers an extensible keyword-driven approach to perform automation testing. The learning curve is simple as you don't need to have programming experience to get started with the Robot framework.

Serverless Eventing Architecture

Posted: 17 Nov 2020 02:47 PM PST

Event-Driven Architecture (EDA) is a way of designing applications and services to respond to real-time information based on the sending and receiving of information about individual events. Serverless is all about providing service on a provision as-used basis. Combining both you get the best of both worlds. 

  • Loose coupling of services - For better fault tolerance, and can add/remove functionality on the go without affecting others. 

Database Drivers: Chauffeuring Your Data to Where it Needs to Go

Posted: 17 Nov 2020 02:41 PM PST

Most, if not all, companies deal with complications and integration headaches somewhere in their data pipeline due to an inability or difficulty of connecting certain systems. Sometimes you have to add yet another technology to the lineup just to connect different systems and get your data to where it needs to go. However, in this modern-day, less is more. Most technologies that emerge are all about being more efficient and providing more functionality in a smaller package. If you can meet your data management needs with fewer tools, then it's a win-win for cost-effectiveness, efficiency, and ease of use. Enter database drivers.

The Magical Adaptors

To put it plainly, each computer system needs some sort of adaptor or tool to be able to connect to other computer systems that are not the same. You can think about this in the physical or interior (computer system) sense. For a physical example, we all know the major frustration that comes along with upgrading your phone, when your chargers and headphones no longer fit in the ever-shrinking port on the bottom of the phone. However, you can buy an adapter to serve as a "middle man" to enable the connection. With computer systems, it's the same but different. A database driver works like that physical phone adaptor, but instead of having to invest in an additional product to add to the tech stack, you can develop an adaptor/connector to extend the database functionality. Like an extension to a software package.

Difference Between InitialRAMPercentage, MinRAMPercentage, MaxRAMPercentage

Posted: 17 Nov 2020 02:36 PM PST

This article attempts to clarify the difference between InitialRAMPercentage, MinRAMPercentage, MaxRAMPercentage JVM arguments. These arguments have been introduced since Java 8 update 191. They are used to configure your Java application's heap size when you are running it in the Physical server or in the container. In this article, let's review their differences.

InitialRAMPercentage

'-XX:InitialRAMPercentage' is used to compute the initial heap size of your java application. Say suppose you are configuring -XX:InitialRAMPercentage=25 and overall physical memory (or container memory) is 1GB then your java application's heap size will be ~250MB (i.e., 25% of 1GB).

Looking for Suitable Tools for Calculating Large Files

Posted: 17 Nov 2020 02:11 PM PST

What is a large file? A large file is a file that is too large to be read in at one time because of insufficient computer memory. In this case, direct use of desktop data tools (such as Excel) is powerless, often need to write a program to deal with it. Even if the program is written, a large file must be read in batches for calculation and processing. Finally, the batch processing results need to be properly summarized according to different calculation types, which is much more complicated than the processing of small file. There are many types of large files, such as text files, Excel files, XML files, JSON files, HTTP files. Among them, text (txt or CSV) is the most common.

The program languages that can be used to process large files are as follows:
1. Conventional high-level programming languages, such as Java, C/C++, C#, Basic, etc.
2. The file data is imported into the database and processed by SQL
3. Python
 4. esProc SPL

What Is ETLT? Merging the Best of ETL and ELT Into a Single ETLT Data Integration Strategy

Posted: 17 Nov 2020 02:02 PM PST

Data integration solutions typically advocate that one approach – either ETL or ELT – is better than the other. In reality, both ETL (extract, transform, load) and ELT (extract, load, transform) serve indispensable roles in the data integration space:

  • ETL is valuable when it comes to data quality, data security, and data compliance. It can also save money on data warehousing costs. However, ETL is slow when ingesting unstructured data, and it can lack flexibility. 
  • ELT is fast when ingesting large amounts of raw, unstructured data. It also brings flexibility to your data integration and data analytics strategies. However, ELT sacrifices data quality, security, and compliance in many cases.

Because ETL and ELT present different strengths and weaknesses, many organizations are using a hybrid "ETLT" approach to get the best of both worlds. In this guide, we'll help you understand the "why, what, and how" of ETLT, so you can determine if it's right for your use-case. 

How to Turn a Raspberry Pi Into a Crypto Trading Bot

Posted: 17 Nov 2020 01:58 PM PST

This tutorial goes through the steps required to install the open-source software that deals with mining data from a crypto exchange, run backtests, and run live trading sessions. The key teaching focuses on setting up the Raspberry Pi the smart way to optimize the use of the limited resources by avoiding running the software GIU on the Pi itself, instead, providing the solution to run the GIU from a regular machine on the Local Area Network.

This Is What You'll Need

  • Raspberry Pi 2GB RAM ($40) - If you plan to run long (multi-year) backtests, you will likely need the 4GB or 8GB version (2 GB is enough for shorter backtests and trading live).
  • Flash-card, 16Gb ($7)
  • Raspberry Pi OS / Raspbian / Ubuntu ($0, open-source)
  • Node.JS ($0, open-source)
  • Git ($0, open-source)
  • Superalgos (0$, open-source)

Let's Start!

I'll assume your Raspberry is fully set up with the native OS or Ubuntu. If it's not, just follow the manufacturer's instructions.

The Rule of Three, SQL Server on Linux Edition

Posted: 17 Nov 2020 01:46 PM PST

When it comes to Microsoft products, the rule of three — at least as far as I'm concerned — is where you can accomplish the same task in three different ways. The go-to example is Microsoft Word, where you can use the ribbon toolbar, a keyboard shortcut, or the context menu to perform the same common formatting options.

As some of you may know, I'm preparing for the upcoming PASS Virtual Summit, where I'm presenting a pre-conference session today titled "SQL Server on Linux from A to Z." As I was preparing my demo scripts, I noticed that this rule of three extends to Linux as well. I've been using Linux (and FreeBSD) since 1996, and have learned in that time that there are many (many!) ways to do the same things.

Remote work and Agility: How to have both under the same roof?

Posted: 17 Nov 2020 01:46 PM PST

The global pandemic has pushed the world to work remotely regardless of the industry type. Even though remote work existed long before the outbreak occurred, it became the new norm or the way of life only after the mandated lockdown. There are umpteen benefits that remote work offers, starting from a better work-life balance to increased levels of productivity. 

However, it's not all rosy. Leaders have and are facing several challenges while leading a distributed team. Communication, the essence of remote work and agility, is one of the major ones. Agility is all about being resilient, adaptive to change and encouraging innovations to boost team efficiency. Fostering an agile environment is difficult in a virtual work set-up, unlike in co-located spaces. 

Starting, Sustaining, and Growing Your Technical Career in a Pandemic

Posted: 17 Nov 2020 01:39 PM PST

On October 15th, Agile Alliance's deliver:Agile Live! session hosted a panel on the unique challenges technical job seekers may be facing during the pandemic.

There have been many requests for a short cheat sheet that summarizes the most important tips and tricks. This is that sheet.

What Should I Put In Front of Hiring Managers (Resume, LinkedIn, GitHub, etc)?

  • Resumes are still the primary thing you use. A bad resume is a no right off the bat.
  • LinkedIn is used heavily by recruiters, but you can mostly copy things over from your (amazing) resume.
  • Make sure you keep those skills really easy to find, like right below your title at each job. Helps everyone know exactly what you've been doing and how current you are.
  • Put your education last, it isn't that important.
  • Your experience sections need to show what outcomes you help with. Don't put that you wrote code. Everyone does that.
  • GitHub, portfolios, and that stuff only gets looked at after they like your resume.
  • Only put things on GitHub or your portfolio that you want to be judged on. One JavaScript error in your portfolio could be the end. An abandoned GitHub repo could convince someone you don't know how to build something
  • At the end of the day, those things might help differentiate you from other new developers, but they're optional.
  • Your previous experience speaks louder than GitHub and portfolios do, so if you're someone like a dev manager or architect lean on your experience more than feeling like you need to start building those things.

What's a Recruiter and How Can They Help?

  • Internal recruiters work for the company itself, and as soon as you talk to them you're being interviewed.
  • External/third-party recruiters are retained by companies to find the best candidate, usually for a fee that is figured as a percent of the first-year salary.
  • Third-party recruiters use the information you give them (e.g. salary requirements, etc.) to help you negotiate the best offer possible.
  • It's important to research who your recruiter is. Look them up and talk to them. Getting a job is a life event, so know who you're working with.
  • A good recruiter is one that will be with you for years, so it's important to build trust.
  • Recruiters live on LinkedIn. They're always looking for people, so having your LinkedIn page mirror your resume helps make sure the recruiters can find you and do their best for you.
  • Recruiters often have exclusive opportunities that will never show up in a job posting.
  • Third-party recruiters also can provide feedback and insight into the process for a particular job, and they keep the interview process moving.

What Are Technical Interviews Like For Developers?

  • There is a proof-of-life, phone screen, several technical interviews, HR, and sometimes leadership.
  • Technical phone screen is usually quick and if you're lucky you'll be asked how you do certain technical things. If you're unlucky you'll be asked obscure, trick, or "gotcha" questions.
  • Technical interviews go deeper and usually involve doing some kind of coding assignment or whiteboarding. It's important to ask questions when you get one of these tasks so you can clarify expectations and tools as well as show you're engaged and thinking.
  • Stories from your past can help you avoid falling victim to a specific technical gotcha while giving the hiring manager a good sense of how you will perform.

What Are Technical Interviews Like For Non-Developers?

  • It varies a lot, unfortunately.
  • You are likely going to be interviewed for the things the person you are backfilling did. If they were a part-time DBA, you'll be asked about it.
  • You'll also get interviewed by someone technical. They're likely to interview you from their own expertise, so it's a good idea to know a bit about their background and current role.

How Should I Go About Negotiating?

  • If you're working with a third-party recruiter, talk to them. A good recruiter will get you the best offer without you realizing it.
  • It's better to say what you want to make instead of saying your previous compensation, but you don't have to throw out a number at all. Keep in mind whatever numbers are discussed will anchor everyone involved.
  • Think about the lifestyle you want before you negotiate. You might be able to get PTO, training, stipends, stock, etc.
  • Ask about all the benefits like healthcare before you start to negotiate. You want to see it all before you counter.

What About Remote Work Because of COVID? Should I Ask For Equipment?

  • Everyone is asking about how to work well remotely. Have a story ready for the question.
  • Yes, it is reasonable and you should ask for equipment to work remotely. Larger companies probably have a package ready. Smaller companies and start-ups you'll have to negotiate.
  • Your recruiter will probably know a bit about this ahead of time and help make sure you get what you need.
  • Using your own equipment can put your intellectual property at risk.

If you have more questions, feel free to reach out to me, Swift Burch III, and Cat Swetel.

How to Integrate a Dialogflow Chatbot With Angular JS

Posted: 17 Nov 2020 01:09 PM PST

There are numerous posts about developing a chatbot using Dialogflow. But creating a chatbot isn't enough. Connecting Dialogflow to the web interface is even more interesting and challenging. With Angular being a popular and emerging platform, here is our guide to integrating Dialogflow chatbot with Angular JS.

In our quest to simplify chatbot integration with different platforms, we have previously written guides to integrating chatbots with websites and apps.

4 Common Reasons for FetchFailed Exception in Apache Spark

Posted: 17 Nov 2020 01:06 PM PST

Shuffle operations are the backbone of almost all Spark Jobs that are aimed at data aggregation, joins, or data restructuring. During a shuffle operation (Without the support of External Shuffle service), the data is shuffled across various nodes of the cluster via a two-step process:

a) Shuffle Write: Shuffle map tasks write the shuffle data to be shuffled in a disk file, the data is arranged in the file according to shuffle reduce tasks. A bunch of shuffle data corresponding to a shuffle reduce task written by a shuffle map task is called a shuffle block. Further, each of the shuffle map tasks informs the driver about the written shuffle data.

Autoscaling Groups With Terraform on AWS Part 2: Instance Security Group and Boot Script

Posted: 17 Nov 2020 12:28 PM PST

Previously, we followed the minimum steps required in order to spin up an autoscaling group in Terraform. In this post, we shall add a security group to the autoscaling group and an http server to serve the requests.

Using our base configuration we shall create the security group for the instances.

Autoscaling Groups With Terraform On AWS Part 1: Basic Steps

Posted: 17 Nov 2020 12:26 PM PST

So you want to create an autoscaling group on AWS using terraform. The following are the minimum steps in order to achieve so.

Before writing the actual code you shall specify the AWS Terraform provider as well as the region on the provider.tf file.

V8 Array Overflow Exploitation: 2019 KCTF Problem 5

Posted: 17 Nov 2020 11:45 AM PST

Introduction to the KCTF Problem

Problem 5 - 小虎还乡 of the 2019 KCTF Competition provides us with a vulnerable v8. The v8 has an array overflow vulnerability. But this is not a native v8 vulnerability. Instead, the authors modified some v8 files and created this vulnerability manually. This post shows you step-by-step on how to exploit this vulnerability.

PoC of the V8 Array Overflow Vulnerability

0   var buggy;  1   var overwrite_length = () => {  2       let oob = new Date(-864000000 * 15000000);  3       oob = Math.abs(oob.getDate() - 16) >> 5;  4       buggy = [1.1];  5       buggy[oob * 4] = 1.1;  6   };  7   for (let i = 0; i < 0x10000; i++) overwrite_length();

The PoC code triggers the array overflow bug. The following explains how the PoC works.

How to Convert ODT Files to PDF in Java

Posted: 17 Nov 2020 11:04 AM PST

Microsoft has maintained its position in the spotlight for formatted document creation and editing for good reason. Its extreme ease of use and lack of a learning curve has transformed the Microsoft Office Suite into a household name for most computer users in the United States as well as globally. This is propagated further through its almost ubiquitous use in education, as students are raised and taught using these applications. 

The issue that arises with these programs, however, is their operation costs. For Apple and other non-Windows-based Operating Systems, the purchasing fees for Office can be steep. This, then, creates a paywall separating potential users from programs to which they are already accustomed. As an answer this problem, Microsoft created the OpenOffice application, which is a free, opensource version of the classic Office Suite. Within this application, you can perform almost all of the same functions as Office Suite, including creating text documents like one would with Microsoft Word. These text documents can be made using OpenOffice Writer, and are formatted using the .ODT file type. While this file type can be opened and saved using OpenOffice Writer and Word, in order to convert the file to a different format such as PDF you will need to run it through a conversion process. 

How to Validate an Email Address in Java

Posted: 17 Nov 2020 11:00 AM PST

Email is one of the most universal tools for sharing and receiving information across the globe, with users able to connect with others online with almost no compatibility or access issues. Using this tool, information can be instantly and securely sent to partners on the other side of the world, and personal information can be verified without divulging sensitive data about a user.

Along with this widespread use, however, comes key security measures that must take place in order to ensure the safety of your organization and data. This is particularly the case when receiving email information from previously unknown sources. These risks can include phishing attempts, malware, and other threats that can cause a negative impact to your business. Furthermore, when receiving an email address via account forms and user sign up information, you need to check that the information you are given is not only correct and real, but also that it does not lead to any malicious sources that could harm your organizational security. 

Python Package for Evaluating Regression Models With Single Line of Code

Posted: 17 Nov 2020 10:58 AM PST

I always believe in democratizing AI and machine learning, and spreading the knowledge in such a way, to cater the larger audiences in general, to harness the power of AI. An attempt inline to this is the development of the python package "regressormetricgraphplot" that is aimed to help users plot the evaluation metric graph with single line code for different widely used regression model metrics comparing them at a glance. With this utility package, it also significantly lowers the barrier for the practitioners to evaluate the different machine learning algorithms in an amateur fashion by applying it to their everyday predictive regression problems. 

Before we dwell into the package details, let's understand a few basic concepts in simple layman terms. 

How to Convert HTML Tables into Beautiful PDFs

Posted: 17 Nov 2020 10:56 AM PST

Web apps that contain tables, charts, and graphs often include an option to export the data as a PDF. Have you ever wondered, as a user,  what's going on under the hood when you click that button?

And as a developer, how do you get the PDF output to look professional? Most free PDF exporters online essentially just convert the HTML content into a PDF without doing any extra formatting, which can make the data hard to read. What if you could also add things like page headers and footers, page numbers, or repeating table column headers? Small touches like these can go a long way toward turning an amateur-looking document into an elegant one.

How to Deploy a Spring Cloud Function on AWS Lambda

Posted: 17 Nov 2020 10:47 AM PST


In this post, we will create a Spring Cloud Function and create some unit tests for it. We will do so by creating a function with Bean definition and with the functional style. At the end, we will deploy the function on AWS Lambda.

1. Introduction

Spring Cloud Function allows us to implement business logic via functions and we can create a Spring Boot application just as we are used to. You can run the function locally but of course it is the intention to run it at some serverless provider. We will show both cases in this post and will use AWS Lambda as serverless provider. You will need to create an AWS account first. See our previous blog How to Deploy a Spring Boot App to AWS Elastic Beanstalk how to create an AWS account. The sources of the examples we are using, can be found at GitHub.

Large Scale Cloud Migrations: Challenges and Recommendations

Posted: 17 Nov 2020 10:45 AM PST

Many enterprises today are adopting cloud in some form or another. While the adoption of cloud is rapid in certain industries like retail, other industries like financial services are taking a slower approach in moving to the cloud due to numerous compliance and regulatory requirements. Enterprises are looking at cloud to migrate their legacy infrastructure and realize the benefits of cloud such as security, scalability and most importantly cost. 

Cloud migrations are complex and require proper planning and execution, almost certainly enterprises will face unexpected challenges during the migration process. To offset some of the challenges organizations can seek help through cloud provider's partner ecosystem or through professional services provided by the cloud provider.

Implementing GraphQL With MuleSoft

Posted: 17 Nov 2020 10:42 AM PST

Introduction

GraphQL is an open-source query language for Rest API's and it is developed by Facebook in 2012 before releasing publicly in 2015. GraphQL allows you to send the query to your API and get the required data. 

REST APIs are powerful and follow a resource-oriented and well-structured approach. However, when the data is complex and it is not possible to fetch all the data within a single request as it involved various operations and routes get more complex. GraphQL is very useful in such cases and structures data in the form of a graph with its powerful query syntax for traversing, retrieving, and modifying data. 

7 Best Angular Component Libraries to Use in 2020

Posted: 17 Nov 2020 10:41 AM PST

Angular is one of the elite JavaScript framework with more than 60,000 GitHub stars. Developed by Google and based on TypeScript, Angular is capable of delivering great web applications with high web page performances.

With consistent updates, Angular has rooted its place in the list of best web development frameworks. Popular brands like PayPal, Upwork, Netflix also use Angular in their frontend development.

Creating a Cluster File Storage With ActiveJ

Posted: 17 Nov 2020 10:39 AM PST

In this tutorial we will create a cluster storage with three servers that communicate via TCP protocol. There also will be a cluster client. Each server will have an HTTP GUI server for simplicity of use and debugging. The cluster storage will support file uploads with automatic repartitioning according to the provided rule and replication count.

What ActiveJ technologies Will Be Used?

What Is ActiveFS?

ActiveFS is one of the most important technologies in this tutorial. It provides a lightweight abstraction on top of common file operations like upload, download, append, list, copy, move, delete, and others. It allows operations with local, remote or distributed file storage. ActiveFS is a one of the ActiveJ Java platform technologies.

Contract First Application Development With Events

Posted: 17 Nov 2020 10:27 AM PST

Introduction

In this post I will go through my demo on how to use the contract first methodology for creating event driven applications. This is going to be a straight up installation instruction, and code with some explanation. I cover most of the basics in the last post. 

This is a simple money transfer application, which receives transfer requests from a restful endpoint.The ultimate goal is to process a single transfer request into transaction records on each account, for better bookkeeping purposes.  The structure of the system is a typical event driven implementation, where you have Topics that are responsible for accepting events, and have services subscribing or publishing to the topics.  In the demo, we have one topic that stores the Transfer Request events, and one stores the Account Record events. 

Static Classes in Java

Posted: 17 Nov 2020 10:19 AM PST

In this article, we are going to study what is a static class in Java and how we can create it. Later, we will discuss some implementation considerations and benefits of using a static class.

Before starting our tutorial on the static class, we first want to briefly remind you that "What is static?",  Static is a keyword that can be used with class, variable, method, and block to create static members. Static members belong to the class instead of a specific instance, this means if you make a member static, .you can access it without an object.

Big Data Certifications in 2021 to Help Start Your Career

Posted: 17 Nov 2020 10:17 AM PST

Breaking into big data industry domain is not an easy nut to crack, especially in contemporary times, wherein the competition is really intense in the job markets for the coveted data science job roles. However, the job openings too, are considerably high in the big data domain, given the dearth of big data professionals across business verticals. 

Data analytics is a booming industry and the job opportunities in the said field of a profession are extensive. As per a Statista study, the worldwide big data analytics market's value was $168.8 billion in 2018, which is expected to rise to a whopping $274.3 billion by the year 2022, thereby registering an impressive CAGR of 13.2%. This year alone, each person on the planet, on average, produced 1.7 MB of data per second. Web-users worldwide, currently are generating 2.5 quintillion bytes of data every single day.  

Packaging With CPack in Nebula Graph

Posted: 17 Nov 2020 10:12 AM PST

CPack is a powerful, easy to use, cross-platform software packaging tool distributed with CMake since version 2.4.2. It helps create binary package and source code package. The following figure shows where CPack locates in the whole CMake tool chain.

CPack supports the following types of package formats:

Impact of Internet of Things on the Small Business

Posted: 17 Nov 2020 10:00 AM PST

Executives and market analysts from nearly every industry say the rise of the Internet of Things (IoT) has been rapid and it revolutionized the world.

Around the world, the number of IoT-connected devices is expected to reach 43 billion by 2023 and a full threefold increase from 2018 according to McKinsey, which will amount to a market forecast to surpass US$176 billion by 2022.

A Journey Through Quality Assurance Testing

Posted: 17 Nov 2020 09:58 AM PST

A while ago, I found myself roaming the new and strange waters of Quality Assurance. Writing tests is a part of the development process, so no matter what technologies are used, sooner or later you learn that testing your code is necessary. While working on Front End, I had to learn really quickly that, "Testing shows the presence, not the absence of bugs." This really got me wondering just how important my role as a QA is in assuring quality and does it rely solely upon me and the QA team to make sure everything works?

In this blog, we will take on a journey exploring the fundamentals of QA, introduce the terms and practices and take a look at two different approaches to development that incorporate testing in the developer's workflow.

Comments

  1. salman khanSeptember 26, 2021 at 8:40 PM

    NZXT Kraken X53 Liquid Cooler in UAE, White 240mm Liquid Cooler in UAE, RGB Fans CPU Liquid Cooler in UAE
    https://gccgamers.com/nzxt-kraken-white.html
    NZXT Kraken X53 in UAE, Safe Shopping Multiple Payment Options Express Delivery GCC Gamers Moneyback Guarantee.
    1632713996115-7

    ReplyDelete

Post a Comment

Popular posts from this blog

DZone.com Feed

Image
DZone.com Feed Kafka on Kubernetes, the Strimzi Way (Part 3) Types of Matplotlib in Python Migration of Java-Based Web Applications From Commercial to Open Source Software Managing Data in the Lakehouse AR and VR Trends: Differences Between Developers and Non-Developers — Part Two How to Make Chatbots More Intelligent With Contextual Intelligence C++ RESTful Web Service With Swagger-UI and Auto-Documented Endpoints DevOps: Release Fast With Artifactory How to Create a Jenkins Declarative Pipeline Promoting APIs and API Implementations to Higher Environments MicroProfile Config With etcd RESTful API Design Driven Approach Optimizing a Simple Ray-Tracer Written in Go Part 2 CircleCI vs Jenkins: Choosing The Right CI/CD Tool Swift Package Manager, Part 1: Introduction to Swift Package Manager Working of Package Manager in Linux Get More Transparency on Your Mi...
Read more