DZone.com Feed

DZone.com Feed


Hands-On GitOps With OneDev and Kubernetes

Posted: 23 Dec 2020 06:21 PM PST

Set Up Kubernetes Cluster

First, we need a Kubernetes cluster. You may use an existing one or set up a new one. For this tutorial, we chose to use GKE (Google Kubernetes Engine).

Just follow the quick start to create a cluster. To save your money, the default pool with only one node is sufficient for our testing. For node image type, use the default Container-Optimized OS; for machine type, please select one with at least 8G mem. After creating the cluster, go ahead to configure kubectl to connect to the cluster following this guide.

Code Your Architecture Structure

Posted: 23 Dec 2020 06:00 PM PST

When you read about software architecture structures and views, you will find different opinions from different authors. And what is worse, they use the same terms for explaining different concepts, which just contributes to the general confusion. 

This article just reviews these concepts and tries to clarify them proposing a simplification.

Key Application Metrics and Monitoring for Developers

Posted: 23 Dec 2020 05:45 PM PST

In the past, code and infrastructure were handled by completely separate organizations. Developers wrote code, while IT set up servers. Developers fixed bugs, while IT handled infrastructure maintenance. However, with the trend towards DevOps and the increased availability of Platform-as-a-Service (PaaS), there is an increasing overlap between Dev and IT. For developers, this can mean taking care of infrastructure, a task that is quite different from standard app dev.

As a full-stack developer, I've been pushed to handle more and more infrastructure-related responsibilities, including monitoring production applications. We just went live for HP Foundation at https://www.life-global.org with our Next.js-based learning management system. As a dev team lead, I have been researching how to best support and maintain the application through metrics.

Embracing NewSQL: Why We Chose TiDB Over MongoDB and MySQL

Posted: 23 Dec 2020 05:43 PM PST

PalFish is a fast-growing online education platform that focuses on English learning. It offers tailored English speaking experience to English as a Second Language (ESL) students. As of October 2020, PalFish has over 40 million users, of which more than 2 million are paid users.

As our business rapidly grew, the surge of data posed a severe challenge to our MongoDB database. MongoDB (2.x and 3.x) does not support transactions and has no predefined schema to directly regulate data. This blocked our business growth. To solve these problems, we migrated from MongoDB to TiDB, an open-source, MySQL-compatible, distributed SQL database that supports Hybrid Transactional/Analytical Processing (HTAP) workloads. This turned out to be the right move.

An Observability Balancing Act With GitOps

Posted: 23 Dec 2020 05:29 PM PST

As companies intensify their push toward adopting DevOps practices and cultural values, there are several practical methodologies cropping up. One such concept is GitOps, which stems from the DevOps need to automate everything and the philosophy of you build it you run it.

The fear of losing out to the competition warrants the need to be agile which then leads to a retrospective of how teams and organizations are getting code from ideation to production. DevOps! The motivation is simple: The faster the release, the more chances of securing the survival of your product in the fast-paced world of technology. 

Python vs PHP for Web Development: Which Is Better?

Posted: 23 Dec 2020 05:15 PM PST

Python and PHP are two extremely popular and high-ranked programming languages. If you may be wondering which language has a greater demand for web development, Python or PHP, Python is the winner. Statistics show that PHP comprises 26.2% of user-share and Python 44.1%. In our article, we will conduct a detailed comparison of these two languages and determine in which cases it is better to use Python or PHP for web development and why Python is more suitable.

Python vs. PHP: Which one to choose for website development in 2021?

Selecting a coding language for any project, be it a website or application development, is a challenging and crucial step. There has always been intense competition between Python and PHP. We will provide a comparison of these two languages for you to make the best choice for your software development. We developed criteria that can be applied to almost any project. Let's get started!

Angular + Spring Boot + PrimeNG Datatable CRUD Example

Posted: 23 Dec 2020 05:05 PM PST

In previous tutorials, we implemented PrimeNG datatable inplace cell editing and PrimeNG datatable row editing. In this tutorial, we will implement a PrimeNG datatable CRUD example. We will use Spring Boot for the backend to expose REST APIs.

Technology Stack

We will be making use of:

Build a Serverless App Using Go and Azure Functions

Posted: 23 Dec 2020 05:04 PM PST

Webhook backend is a popular use case for FaaS (Functions-as-a-service) platforms. They could be used for many use cases such as sending customer notifications to responding with funny GIFs! Using a Serverless function, it's quite convenient to encapsulate the webhook functionality and expose it in the form of an HTTP endpoint. In this tutorial, you will learn how to implement a Slack app as a Serverless backend using Azure Functions and Go. You can extend the Slack platform and integrate services by implementing custom apps or workflows that have access to the full scope of the platform allowing you to build powerful experiences in Slack.

This is a simpler version of the Giphy for Slack. The original Giphy Slack app works by responding with multiple GIFs in response to a search request. For the sake of simplicity, the function app demonstrated in this post just returns a single (random) image corresponding to a search keyword using the Giphy Random API. This post provides a step-by-step guide to getting the application deployed to Azure Functions and integrating it with your Slack workspace.

Why Migrate to Kubernetes in 2021: Key DevOps and DataOps Benefits

Posted: 23 Dec 2020 04:51 PM PST

Since its release in 2015 followed by multiple commits on GitHub, Kubernetes (or K8s) has been scoring points across industries and professional communities. But while the reasons behind the success of Kubernetes are crystal clear for software developers, DevOps, and datascience engineers, it is not that obvious for business owners.

Why has transferring a company's application-related processes to Kubernetes has become a mainstay? To find this out, let's go over and brush up on some key concepts driving the benefits K8s can bring about.

How Do You Delete a Local Branch in Git?

Posted: 23 Dec 2020 04:37 PM PST

At its core, the branching model offered by Git is intended to help you avoid merging unstable code into the main codebase. Working with branches in Git is a breeze, especially if you're working with the GitKraken Git client to visualize your repository.

How to Delete a Git Branch Locally Using the Command Line

To delete a local branch in Git using the terminal, you're going to run the git branch command and pass in the -d flag. Next, you will pass in the name of the branch you wish to delete.

10 Best IoT Platforms for 2021

Posted: 23 Dec 2020 04:19 PM PST

The rapid growth of the Internet of Things over recent years has brought countless examples of smart devices, cloud services, and applications to life. As their complexity increases, the need for IoT platforms   ̶  solution frameworks that streamline IoT application development   ̶  also grows. In their recent report, MarketsandMarkets estimated that the global IoT cloud platform market will grow from USD 6.4 billion in 2020 to USD 11.5 billion by 2025, at a Compound Annual Growth Rate (CAGR) of 12.6% during the forecast period.

The number of IoT platform vendors shows similar dynamics; as estimated by Statista, there were about 620 publicly known IoT platforms in 2019, twice as many as in 2015. Big companies from different industries, such as Amazon, Bosch, and Deutsche Telekom, have joined the fray, all trying to increase their footprint in technology markets. However, a bigger vendor doesn't always mean a better solution. When choosing the best IoT platform for your business, it is important to consider the total cost of ownership, usability, speed of implementation, etc. as much as strong feature sets. What is considered to be a great IoT platform, in general, may not be the best option for your case.

Agile Best Practices for Remote Work Excellence

Posted: 23 Dec 2020 04:19 PM PST

Introduction

The COVID-19 virus has disrupted peoples' lifestyles and the working patterns of many sectors and industries. The IT industry has seen a thorough transformation within the premises, yet no significant changes in the outputs. Here in this blog, let me consider the discrepancies that working remotely has caused the IT companies, especially the Agile software development teams. 

The prime role of Agile is to aid the co-located team function effectively. However, the pandemic has obliged everyone to work from home, i.e., working remotely, and the on-floor teams got converted into distributed teams. 

Book Review: Foundations of Deep Reinforcement Learning, by Laura Graesser and Wah Loon Keng

Posted: 23 Dec 2020 04:08 PM PST

Deep Reinforcement Learning is a somewhat new field within Machine Learning or Artificial Intelligence (you may pick your favorite term between these two, even if they're not strictly the same), which combines Deep Learning and Reinforcement Learning and is based on the general idea that an agent can learn by observing its actions and their consequences. No, it is not a return to John B. Watson and B. F. Skinner's behavioral psychology. We are talking, instead, about a set of pretty advanced machine learning algorithms that, when properly mastered, allow computers to achieve remarkable results in some complex tasks. That's what this book is about, so let's dive in…

Book Structure and Contents

Foundations of Deep Reinforcement Learning - Theory and Practice in Python begins with a brief preliminary chapter, which serves to introduce a few concepts and terms that will be used throughout all the other chapters: agent, state, action, objective, reward, reinforcement, policy, value function, model, trajectory, transition.

FaaS: Security Considerations to Know Before Going Serverless

Posted: 23 Dec 2020 03:51 PM PST

Serverless architecture is becoming a compelling choice for developers and companies to host their applications. It is easy to see why with its ability to dynamically scale to meet load requirements as well as removing a lot of the complexity with deploying and maintaining applications, sometimes even removing the need for an Ops team. But what are the security considerations we should consider before choosing to go serverless?

What is Serverless Architecture?

Serverless architecture (also known as serverless computing or function as a service, FaaS) is a software architecture where applications are hosted by a third-party service. This essentially means that your application is broken into individual services, which negates the need for server software and hardware management by the developers.

How to Create a Heatmap in Pivot Grids

Posted: 23 Dec 2020 03:32 PM PST

The ways and levels of perception have always differed from person to person. Although, in our time, most of the information is absorbed visually. This is the main reason why, for a better understanding of the user's content, you need to choose the right type of data visualization. Here, I want to talk about one of the ways of displaying information in tables; heatmaps. 

What is a heatmap?

A heatmap is a graphical representation of data where values of quantitative variables are expressed as colors, mainly by hue or intensity, giving obvious visual cues about how the phenomenon is clustered or varies over space. 

Ingest Data From Apache Kafka Into Azure Cosmos DB Cassandra API

Posted: 23 Dec 2020 03:29 PM PST

Azure Cosmos DB Cassandra API is a fully managed cloud service that is compatible with Cassandra Query Language (CQL) v3.11 API. It has no operational overhead, and you can benefit from all the underlying Azure Cosmos DB capabilities such as global distribution, automatic scale-out partitioning, availability and latency guarantees, encryption at rest, backups, etc.

At the time of writing this blog, the Azure Cosmos DB Cassandra API serverless is available in preview mode!

Modular Monolith vs Upfront Microservices

Posted: 23 Dec 2020 03:07 PM PST

Not all monoliths are bad. The above picture is from a trip I took to the Monolith Temples in Mahabalipuram, India. But how does this relate to software engineering?

Finding (micro)service boundaries is a challenge and become relatively easy as we learn more about the business domain. So, we wanted to try a modular monolith (modules within the microservice) approach.

As the module matures and we see it becoming the first-class microservice, we can take it out of the modular monolith very easily and with fewer efforts as the module is developed as a first-class concept with loose coupling with other modules.

Lessons Learned from the November AWS Outage

Posted: 23 Dec 2020 02:22 PM PST

Context, Analysis, and Impact

  • Amazon's internet infrastructure service experienced a multi-hour outage on Wednesday, November 25th, that affected a large portion of the internet.
  • More than 50+ companies were impacted, including Roku, Adobe, Flickr, Twilio, Tribune Publishing, and Amazon's smart security division, Ring, in its region covering the eastern U.S.
  • Business impacts, as reported by The Washington Post, included:
    • New account activation and the mobile app for streaming media service Roku became hampered.
    • Target-owned Shipt delivery service could receive and process some orders, though it stated that it was taking steps to manage capacity because of the outage.
    • Photo storage service Flickr tweeted that customers couldn't log in or create an account because of the AWS outage.

Tweets by companies experiencing outages.

  • Root Cause Analysis by AWS: It started with Amazon Kinesis but started impacting a long list of services. You can read the RCA document by AWS, which is also summarized below:
    Flowchart of AWS impact spread.

Lessons Learned

#1: Don't Put All Your Eggs in One Basket

  • Using a single Cloud Service Provider can be counter-productive in these scenarios.
  • Think and strategize for Hybrid-Cloud or Private Cloud; or Multi-Cloud, particularly during peak season.

#2: Hope for the Best and Plan for the Worst

  • Don't just rely on a cloud provider's availability and multi-region fail-over strategy; build your own resiliency and disaster recovery approach.
  • Practice disaster recovery in production or similar systems by using innovative approaches in active-active setup across the multi-cloud or hybrid-cloud scenarios.

#3: Monitoring and Observability Are Not Static

  • Be innovative in exploring monitoring and observability patterns. For example, if AWS is reporting an outage on their status page, your monitoring system should get into action and inform the incident resolution team to start analyzing the impact.
  • Keep ready the services dependency graph; though mostly supported by tools, you should keep it dynamic and prepared to assess the impact when it happens and map it to business functionalities to report it to your business team accurately.

#4: Invest in Emerging Techniques, like Chaos Engineering

  • This failure indicates that even internet giants like AWS are still maturing in implementing practices like chaos engineering. So, start putting chaos engineering practices into the roadmap.
  • For example, if a bulkhead pattern could have been utilized in the AWS outage scenario, the outage would have been limited to Kinesis services only.

To conclude, being proactive when outages occur, having a response team equipped for unplanned outages, and improving continuously from lessons learned along the way are essential techniques to help keep the impact limited. Also, having a multi-cloud or hybrid-cloud strategy is food for thought to keep the business running.

The Limitations of Automation

Posted: 23 Dec 2020 01:55 PM PST

Knowing what you can and can't automate is important, but the real question is: Can you automate yourself out of a job? 

On this week's episode I brought in tenured software engineer and automation expert Judy Johnson to answer my questions about automation. In this episode Judy helps me answer questions like: 

Java and Hibernate: Entity Management in Multi-Client, Single Codebase Apps

Posted: 23 Dec 2020 12:43 PM PST

When developing an application for only one client or only a single version, the main subjects you focus on are how to implement the features, security, performance, design, time to market, etc. What will happen if the number of clients increases, and each one of them has different requests that contradict each other? Are we going to duplicate our codebase and have different apps, which we have to implement common features to every instance separately? Or are we going to manage them from one codebase?

Diversity in labels might be managed with different property files, but will this be enough for all kinds of requests? The flow in the logic might be directed with control blocks. What will happen to them when you have 100 different clients? What will happen to the readability/manageability of the code? We might decide to extract the logic that differs from one client to another and have multiple microservices. This will allow us to have clean code, but will we be able to turn all the logic that differs from client to client into microservices? Is our structure suitable for this process? How about managing and deploying all those services? Is it worth the cost?

Understanding Kubernetes Operators

Posted: 23 Dec 2020 12:17 PM PST

Automation is one of the fundamental components that makes Kubernetes so robust as a containerization engine. Even complex cloud infrastructure creation can be automated in order to simplify the process of managing cloud deployments. Despite the capability of leveraging so many resources and components to support an application, your cloud environment can still be fairly manageable.

Despite the many tools available on Kubernetes, the effort to make cloud infrastructure management more scalable and automated is ongoing. Kubernetes operator is one of the tools designed to push automation past its limits. You can do so much more without having to rely on manual inputs every time. 

Using Shipa to Automate Kubernetes Object Creation

Posted: 23 Dec 2020 12:01 PM PST

If you're like most developers, you don't have a ton of spare time (and perhaps also not much desire) to learn how to create and maintain Kubernetes objects. Where developers are required to manage Kubernetes, slower application delivery is often the result. In these scenarios, platform and DevSecOps teams must also work to support developers' Kubernetes efforts, and cope with ever-greater chances that deployed applications will suffer from misconfigured objects. Platform teams are usually tasked with creating an extra platform layer as well, adding to costs and making maintenance and scaling more difficult (while also increasing the potential for failure).

Take a scenario in which you deploy a single 3-tier application. This requires maintaining a dozen Kubernetes objects, creating a container image of your application, and optimizing the container OS for storage, security, compliance, and more. Enterprise IT must carefully consider the time it takes for developers to deploy and operate the application, scale clusters, and create Kubernetes objects and YAML files. They must also factor in the tools they use for security monitoring, the frequency with which they must write Ansible and Terraform scripts, and whether they have time to focus on cloud infrastructure APIs.

SOAP Transformation With Apache Camel Java DSL

Posted: 23 Dec 2020 11:45 AM PST

One of the most common issues that we usually tackle with customers when modernizing legacy integrations is when they have a collection of legacy SOAP web services that don't integrate well enough with the modern RESTful-based clients.

In this article, we are going to use Apache Camel to build a REST endpoint that will translate REST calls to a SOAP envelope, get the response, and send it back to the client.

Troubleshooting a Crash Triggered by Clang Compiler Optimization

Posted: 23 Dec 2020 11:25 AM PST

If someone told you that the following C++ function would cause the program to crash, what would you think it is that caused the problem?

C++
 




xxxxxxxxxx
1


 
1
std::string b2s(bool b) {
2
    return b ? "true" : "false";
3
}



How to Keep Software Testers Motivated

Posted: 23 Dec 2020 10:46 AM PST

Recently, I gave a talk at Agile Testing Days called "How to Keep Testers Motivated." This topic is one that's close to my heart because as COO, I manage all of Abstracta's operations that rely on the effort of more than 100 software testers. Therefore, the motivation of the people working with us is a key factor to success: the company's success, the different teams' success, and their personal success.

I remember last year during the TestingUY conference in Uruguay, Melissa Eaden was answering a question from the audience and she said that everyone can do testing, but only a tester does good testing. I fully agree with her, but I like to add a little detail to this affirmation, which is that only a motivated tester does good testing.

Multi-Protocol File Transfer API Integrations in Python Using Zato

Posted: 23 Dec 2020 10:44 AM PST

In many domains, the transfer of static and batch files is an important part of systems integrations and a large number of applications produce and expect data in the form of files rather than network-based APIs. In this article, we shall see how Zato makes multi-protocol integrations of this kind possible in a way that is secure, scalable and easy to extend in Python.

File transfer is often found in scenarios such as:

Some Advice for WFH Agile

Posted: 23 Dec 2020 10:27 AM PST

Introduction

The now popular Agile methodology, with Scrum methodologies being popular cousins to Agile, has taken off since the publication of the Agile Manifesto in 2001. Many things changed with the coronavirus, however, as remote working became one solution for the current problem. This imposed changes in the current approach of managing Agile teams. Since these teams work based on close groups and one location, the main challenge is to keep the same productivity level in a sudden transition. 

Agile teamwork enables direct contact, immediate communication, and problem solving. This will speed up the decision-making process, allowing for better fact-based decisions. Agile teams that don't have experience with remote working will find the sudden change difficult. The new approach will decrease cohesion and, therefore, obstruct the whole process. 

Top 6 JavaScript Family Tree Diagram Libraries

Posted: 23 Dec 2020 07:20 AM PST

A family tree is a type of diagram used for showcasing a person's ancestry and generational bonds in a compelling tree-like structure. This data visualization tool is widely utilized in historical, genealogical, social, or medical web projects.

Building a well-structured and informative JavaScript family tree diagram from scratch and integrating such functionality into an existing web app can be a challenging task. That is why web developers frequently rely on JS components to speed up the process and avoid unnecessary bugs. 

What Is CIDR (Classless Inter-Domain Routing) in MuleSoft VPC

Posted: 23 Dec 2020 05:39 AM PST

Introduction

CIDR stands for Classless Inter-Domain Routing and it way of allocating IP address or host in more efficient manner. It replaces the old way of allocating IP address on based of class system. This method allocate the IP Addresses or host in more efficient way and avoid waste of IP Addresses.

  • Class A, 16 million IP Addresses.
  • Class B, 65,536 IP Addresses.
  • Class C, 256 IP Addresses

Let's consider the Organization requiring around 500 IP Addresses. In such cases, organizations have to go with a Class B IP distribution system where almost more than 60,000 IP addresses are wasted.

Comments

Popular posts from this blog

DZone.com Feed