Discussing Data Models and Data Flows

It's often joked that being a backend dev is just about storing and sending data, but in reality, it's a crucial aspect of how technology functions.

For backend developers, managing data is a big part of their job. We will discuss two commonly used data models and data flow standards.

Data Models

• JSON
• Protobuffers

Data Flows

• REST
• RPC

JSON

JSON, or JavaScript Object Notation, was created to streamline the exchange of data between different systems, particularly web servers and browsers. It's designed to be easily readable by both humans and computers, using a simple key-value pair format.

One of its main advantages is its lightweight nature, which means it doesn't consume much memory or processing power as compared to traditional data models like XML.

In essence, JSON simplifies data exchange across various platforms, making it a popular choice for developers working on web-based projects.

JSON uses text-based characters, which can take up more memory than binary formats. However, it can be converted into binary strings for more efficient storage and transmission, using methods like BSON, to reduce memory usage and improve performance in data processing.

Proto

Protocol Buffers (Proto) is a method for serializing structured data in a compact, efficient, and platform-independent manner. It works by defining a structure for the data, which is then converted into a compact binary format. This binary format saves memory and is speedy because it's designed to be small and easy for computers to process quickly

In this example, we define a message type called "Person" with fields for name, age, and a list of scores.

When data is serialized using Proto, it's converted into a byte sequence according to the defined schema. Each field is encoded along with its field number and type information, resulting in a compact binary representation.

For example, serializing a "Person" object with name="John", age=30, and scores=[85, 92, 78] might result in a byte sequence like this:

08 4A 6F 68 6E 10 1E 18 55 18 5C 18 4E        

In this byte sequence:

• "08" represents field number 1 (name), followed by the length of the string (4 bytes), and the ASCII representation of "John".
• "10" represents field number 2 (age), followed by the encoded integer value 30.
• "18" represents field number 3 (scores), followed by the length of the repeated field (3 bytes), and the encoded integer values 85, 92, and 78.

Key Features of Proto:

1. Compact Binary Format: Proto uses a compact binary encoding that minimizes the size of the serialized data, reducing memory usage and network bandwidth requirements.
2. Efficient Encoding: Proto efficiently encodes data types and field numbers, avoiding unnecessary overhead and ensuring fast serialization and deserialization.
3. Code Generation: Proto generates optimized serialization and deserialization code in various programming languages, tailored for performance and efficiency.
4. Platform-Independent: Proto's binary format is platform-independent, allowing data to be serialized and deserialized efficiently across different systems and programming languages.

REST

REST (Representational State Transfer) is an architectural style for designing networked applications, like websites or web services. It uses standard HTTP methods (like GET, POST, PUT, DELETE) to perform actions on resources (such as data or files) identified by URLs. RESTful systems are built around the idea of treating data as resources that can be accessed and modified using a uniform and predictable interface, making it easy to interact with and integrate different parts of a system or application.

Read: How I explained REST to my wife... , it explains the Rest in the simplest way possible.

RPC

Remote Procedure Call (RPC) is a way for different programs or processes to communicate with each other over a network. It works by allowing one program to call a function or procedure in another program as if it were a local function call. The calling program sends a request to the remote program, which then executes the requested function and returns the result. RPC simplifies communication between distributed systems by abstracting away the complexities of network communication, allowing developers to focus on the logic of their applications rather than the details of how data is transmitted between them.

Traditional RPC has problems like needing both client and server to use the same language, which can be restrictive. It can also slow things down and make defining services complicated.

Here comes the GRPC: that fixed these issues. It lets the client and server talk in different languages, making it more flexible. GRPC makes it easier to define services using Protocol Buffers, a simple way to describe data and functions. And it uses HTTP/2, which helps it run smoother and faster.