distributed sql

YugabyteDB

YugabyteDB, What Is It Good For?

YugabyteDB, What Is It Good For? 800 507 Exist Software Labs

The database evolved from the network & hierarchical model of the 60s to what eventually became the dominant force in databases in the 70s up to the present time—the relational model. From this model arose the big names in Relational Database Management Systems like DB2, Oracle, and Sybase. In the 80s, a player emerged that would forever change the RDBMS landscape: Postgres.

Postgres is the only RDBMS to have won DB of the year 3 times: 2017, 2018, & 2020. This speaks of the universal trust that developers have placed on the system and this has only increased with the rise of open-source software and cloud-native computing.

Which begs the question: What is cloud-native computing? To put it simply, the rise in the services-based model championed by the major cloud providers like AWS, GCP, & Azure has given way to a new way of developing applications, with key emphases on scalability, resilience, high-availability, and agile deployments.

Combining the relational supremacy of Postgres and cloud-native computing, we are then faced with the next step in the evolution of the RDBMS—Distributed SQL— and YugabyteDB is the No. 1 Distributed SQL platform today.

Use Cases

What is YugabyteDB good for? I will mention 4: cloud-native applications, applications requiring massive scale, geo-distributed workloads, and traditional RDBMS modernization.

1. Cloud-native applications – Build stateful microservices by leveraging modern cloud-native frameworks such as GraphQL, or in any language of your choice like Django, Spring, Go, etc.

2. Massive-scale applications – Seamlessly deploy IoT and streaming services that demand high throughput, support for large data sets, and/or many concurrent connections.

3. Geo-distribution – Move data closer to users for resilience, performance, and compliance with the industry’s largest choice of deployment and replication models.

4. RDBMS modernization – Evolve Oracle, SQL Server, and DB2 workloads to a modern distributed SQL database with minimal disruption. If you can migrate to PostgreSQL, you can migrate to YugabyteDB.


Real-World Projects

We have deployed YugabyteDB in several government and educational institutions that employed cloud-native applications development—requiring user connectivity from various geo-locations—along with the migration of MySQL databases.

YugabyteDB has delivered in all of these projects, requiring very minimal tweaking in existing SQL code and even speeding up queries in many instances.

YugabyteDB will very soon be the de facto RDBMS of choice in a cloud-native world.

Exist is your data solutions partner of choice!

Explore the next level of your digital transformation journey with our Data Solutions Services. Let’s look at opportunities to better maximize your ROI by turning your data into actionable intelligence. Connect with us today, and we’ll proudly collaborate with you!

The Elephant Behind the Excellence

The Elephant Behind the Excellence 650 486 Exist Software Labs

The preceding year, 2021, was an eventful year for EXIST Data Solutions: new team members were added, new technologies were learned, and new projects were implemented.

In the enterprise database front, PostgrEX was implemented in a prestigious 5-star hotel and casino, a state university, and a major security agency handling the biggest mall in the country.

YugabyteDB, the No. 1 cloud-native, distributed SQL database in the world, was implemented in 3 government projects, and premium EnterpriseDB support was rendered to the country’s primary energy market corporation.

On the data analytics front, Greenplum was also successfully implemented in 3 government projects, thereby enabling these entities to turn their data into actionable insights.

But what do all these business-transforming technologies have in common? In a word: Postgres.

Postgres is the database engine upon which PostgrEX, YugabyteDB, EDB, and Greenplum are based. With most of them, modifications in varying degrees were done to core Postgres to deliver a product that is still Postgres, but better!

As indicated in the article, Databases in 2021: A Year in Review, the dominance of Postgres in the year 2021 was undeniable:

The conventional wisdom among developers has shifted: PostgreSQL has become the first choice in new applications. It is reliable. It has many features and keeps adding more. In 2010, the PostgreSQL development team switched to a more aggressive release schedule to put out a new major version once per year (H/T Tomas Vondra). And of course, PostgreSQL is open-source.

PostgreSQL compatibility is a distinguishing feature for a lot of systems now. Such compatibility is achieved by supporting PostgreSQL’s SQL dialect (DuckDB), wire protocol (QuestDB, HyPer), or the entire front-end (Amazon Aurora, YugaByte, Yellowbrick). The big players have jumped on board. Google announced in October that they added PostgreSQL compatibility in Cloud Spanner. Also in October, Amazon announced the Babelfish feature for converting SQL Server queries into Aurora PostgreSQL.

One measurement of the popularity of a database is the DB-Engine rankings. This ranking is not perfect and the score is somewhat subjective, but it’s a reasonable approximation for the top 10 systems. As of December 2021, the ranking shows that while PostgreSQL remains the fourth most popular database (after Oracle, MySQL, and MSSQL), it reduced the gap with MSSQL in the past year.

Another trend to consider is how often PostgreSQL is mentioned in online communities. This gives another signal for what people are talking about in databases.

What does all this mean for you and your business? It means you can entrust your most mission-critical applications to Postgres and its derivatives; it means you can break free of vendor lock-in and redirect cost savings to core business initiatives; it means your company can be a better version of itself–a more profitable version–in the year 2022!

Contact us and find out how EXIST Data Solutions can meet all your database-related requirements.

Exist is your data solutions partner of choice!

Explore the next level of your digital transformation journey with big data and analytics. Let’s look at opportunities to better maximize your ROI by turning your data into actionable intelligence. Connect with us today, and we’ll proudly collaborate with you!

A Fully Dockerized MySQL to YugabyteDB Migration Strategy Using pgloader

A Fully Dockerized MySQL to YugabyteDB Migration Strategy Using pgloader 768 487 Exist Software Labs

While there have been many who began their journey to relational databases with the simple and popular MySQL, the evolution of business use cases involving more than read optimization and the need for more performant, full-fledged, read/write-optimized OLTP systems have given rise to a widespread migration from MySQL to Postgres.

Along with this, the transition from monolithic to cloud-native has also paved the way for distributed SQL systems that allow for read/write functionality in every node of the database cluster (while maintaining ACID-compliance across all nodes) and cloud-agnostic deployments of these nodes across geographic zones and regions. This is the future of the database, a future where reliability, accessibility, and scalability are built into the product. The future of the database is YugabyteDB.
 

From MySQL to YugabyteDBfast!

The method that we will be using to migrate a MySQL database to YugabyteDB is through the use of pgloader, a very reliable tool for migrating from MySQL (even SQL Server) to Postgres. We will first migrate the MySQL database to a Dockerized Postgres instance using Dockerized pgloader.

Once the MySQL database has been migrated to Postgres, we will then use the ysql_dump utility that comes with every installation of YugabyteDB to dump the Postgres database into a YugabyteDB-friendly format. This is one of the very useful traits of ysql_dump: it ensures that your Postgres dump can be fully restored in a YugabyteDB instance.

After getting the dump, we will restore this dump in the blank YugabyteDB database that we’ve created beforehand, thereby completing the migration from MySQL to YugabyteDB!

 

Steps

1. Get the Postgres Docker container

docker run -e POSTGRES_HOST_AUTH_METHOD=trust -p 5432:5432 -d postgres:11

2. Create the MySQL database counterpart in Dockerized Postgres

CREATE DATABASE <db name>;

3. Run Dockerized pgloader to load from MySQL to Dockerized Postgres

docker run --rm --name pgloader dimitri/pgloader:latest pgloader --debug mysql://<user name>:<password>@<ip address of MySQL DB server>:3306/<source database name> postgresql://postgres@<ip address of Dockerized Postgres>:5432/<destination database name>

*If a user error is encountered, make sure the user and IP address combination indicated in the error is created in the MySQL source and has access to the databases to be migrated.”

4. Since pgloader creates a Postgres schema using the database name and puts the tables there, we can change the schema name to “public”

DO LANGUAGE plpgsql
     $body$
     DECLARE
     l_old_schema NAME = '<schema name>';
     l_new_schema NAME = 'public';
     l_sql TEXT;
     BEGIN
     FOR l_sql IN
     SELECT
          format('ALTER TABLE %I.%I SET SCHEMA %I', n.nspname, c.relname, l_new_schema)
     FROM pg_class c
          JOIN pg_namespace n ON n.oid = c.relnamespace
     WHERE
     n.nspname = l_old_schema AND
     c.relkind = 'r'
     LOOP
     RAISE NOTICE 'applying %', l_sql;
     EXECUTE l_sql;
     END LOOP;
     END;
     $body$;

5. In this example, we will be using Dockerized Yugabyte as the destination (also applies to other form factors)

a. 1-node cluster with no persistence: 

docker run -d --name yugabyte  -p7000:7000 -p9000:9000 -p5433:5433 -p9042:9042 yugabytedb/yugabyte:latest bin/yugabyted start --daemon=false

b. With persistence:

docker run -d --name yugabyte  -p7000:7000 -p9000:9000 -p5433:5433 -p9042:9042 -v ~/yb_data:/home/yugabyte/var yugabytedb/yugabyte:latest bin/yugabyted start --daemon=false

6. Go inside the Yugabyte container

a. To access the interactive terminal of the container:

docker exec -it <yugabyte container id> /bin/bash

b. Go to the bin directory:

cd /home/yugabyte/postgres/bin

c. Make sure destination database exists in YugabyteDB:

CREATE DATABASE <destination yugabytedb name>;

d. Dump the database in the Postgres container:

./ysql_dump -h <ip address of Postgres container> -U postgres -d <database name of postgres db> -p 5432 -f <dump name>.sql

e. Restore the Postgres dump in the blank database in the YugabyteDB instance:

./ysqlsh -p 5433 -d <database name of destination yugabyte db> -f <dump name>.sql

 

And there you have it! You have successfully migrated your MySQL database to the future of the database. You have migrated to YugabyteDB!

yugabytedb migration

Exist is your data solutions partner of choice!

Explore the next level of your digital transformation journey with big data and analytics. Let’s look at opportunities to better maximize your ROI by turning your data into actionable intelligence. Connect with us today, and we’ll proudly collaborate with you!

Enterprise Database

The Future of the Database: YugabyteDB

The Future of the Database: YugabyteDB 768 487 Exist Software Labs

The journey to application modernization brought about by the cloud-native renaissance continues, and the benefits to be had are truly being enjoyed by the enterprises that embrace the path. Speed, scalability, resiliency, and agility may seem to just be industry buzzwords, but in reality, they translate to better application deployment, performance, and availability, which further translate to what really matters: happy customers.

This has given way to the concomitant need for databases to adapt to this need for speed, scalability, resiliency, and agility. The way traditional databases have implemented a single-node access to the database cluster via the master node has proven untenable in a commercial environment wherein the need to scale users, not just locally, but across the regional and geographical divide, has become dire and ubiquitous.

This is where the gap is filled by YugabyteDB.

 

What is YugabyteDB?

What is YugabyteDB?

YugabyteDB is a transactional, distributed SQL database that was designed primarily to possess the virtues of the cloud-native philosophy. Its creators wanted a chiefly OLTP database that was fast, easy to add more nodes to, able to tolerate node failures, upgradable without incurring any downtime, and deployable in all form factors (public/private cloud, VMs, and on-prem).

Being a distributed SQL database, it has automatic distribution of data across nodes in a cluster, automatic replication of data in a strongly consistent manner, support for distributed query execution so clients do not need to know about the underlying distribution of data, and support for distributed ACID transactions.

It is a multi-API database that exposes the following APIs (more will be added in the future): 

  • YSQL – an ANSI SQL, fully-relational API that is completely compatible with PostgreSQL 11.2
  • YCQL – a semi-relational SQL API that is based on the Cassandra Query Language

It is a Consistent and Partition Tolerant (CP) database in that in the event of a network partition within the database cluster wherein one of the nodes cannot communicate with the other nodes and determine majority membership, data consistency over availability is prioritized by the system and this node will not be able to accept writes, whereas the nodes that are still part of the majority will remain unaffected.

It is completely open source, released under the Apache 2.0 license.

 

What are the key benefits of YugabyteDB?

The following are some of the benefits that are immediately enjoyed “out-of-the-box”:

  • No single point of failure given all nodes are equal
  • Distributed transactions across any number of nodes
  • Scale write throughput linearly across multiple nodes and/or geographic regions.
  • Low-latency reads and high-throughput writes.
  • Strongly consistent, zero data loss writes.
  • Cloud-neutral deployments with a Kubernetes-native database.
  • Automatic failover and native repair.
  • 100% Apache 2.0 open source even for enterprise features.

In other words, you get a cloud-native, transactional, distributed SQL database system that allows you to read and write on every node in the cluster (with ACID assurance), distribute your application load across many nodes in many regions and geographies, read and write data fast, deploy anywhere, and be highly available—all in open source!

 

Use Cases

YugabyteDB is perfect for:Use Cases of YugabyteDB

Just this morning, social media personality, James Deakin, posted on his FB wall about a particular bank whose “app feels like it’s running on windows 95” (his own words). He ended up closing his account due to the overall poor customer experience brought on by the subpar performance of this bank’s client-facing, internet applications, along with other concerns.

YugabyteDB is perfect for the client-facing, Internet, transactional application.

Want to know more about the Yuggernaut of Distributed SQL? Contact us.

Exist is your data solutions partner of choice!

Explore the next level of your digital transformation journey with big data and analytics. Let’s look at opportunities to better maximize your ROI by turning your data into actionable intelligence. Connect with us today, and we’ll proudly collaborate with you!