Day-21: Implementing HTTP Request Logs and MongoDB in Express.js
#LearningExpressJS #Day21
On day 21 of learning Express.js, I learned about HTTP Request logs, their importance in server-side (backend) development, and how to implement them. I also got familiar with MongoDB, a NoSQL database. I learned about the main components and hierarchy of MongoDB, including projects, clusters, databases, and collections. I understood these components and explored useful packages like Morgan and Mongoose. Today, I connected the MongoDB database with an Express server. Here is the bundle of concepts I learned today:
HTTP Request Logs
HTTP logs are a very important component in backend server-side development, especially when working with frameworks like Express.js. Also, http logs known as server logs or access logs, these logs capture valuable information about the requests made to the application. This includes details such as what the user requested, how the request was made, when the user sent the request, how long it took to respond, which response was sent according to the request, the request method used, and any changes made by that request on the server side.
Capturing these logs is essential for creating dynamic and quick-responsive applications. By analyzing HTTP logs, we can assess the application's performance and response rate. This analysis helps in choosing better algorithms to make the server side faster and more responsive, as well as in error detection and handling, data flow control, memory management, CPU management, and more.
Therefore, logs are an essential part of server-side development, crucial for monitoring and troubleshooting, security measurement, and creating high-quality applications.
Implementing HTTP Logs in Express.js
Previously, if developers wanted to track logs, they had to manually code an HTTP logger. This involved defining a function that would take in the request and response objects, then log the desired details. The logging function was invoked using middleware for each incoming request.
However, nowadays, there are many packages available that simplify HTTP log implementation. These packages help developers monitor, record, and analyze log data without the need for manual coding. They offer various features, including log formatting, log rotation, remote logging, and integration with logging services. Popular packages for logging in an Express server include Winston, Morgan, Bunyan, and Log4js.
In our session, we focused on implementing the "Morgan" package for logging. By adding Morgan to an Express app, we can automatically log every request that hits our server. This middleware can be configured to log in various formats, from simple output to detailed logs that include response time and headers.
Here's a basic example of how to set up Morgan in an Express application:
Step 1: Install Morgan Package
First, we opened our terminal and navigated to our project directory. We then installed the Morgan package using npm:
npm install morgan
Step 2: Require Morgan in Your Server File
Next, we opened our “server.js” file (or the main server file of our Express project) and required the Morgan module at the top of the file:
const morgan = require('morgan');
Step 3: Use Morgan Middleware in Express
We then added Morgan as middleware in our Express application. We typically used the 'dev' format for logging, but we could choose other formats such as 'tiny' or 'combined' based on our needs:
const express = require("express");
require("dotenv").config();
const app = express();
const port = process.env.PORT ?? 8000;
const morgan = require("morgan");
// Middleware to log HTTP requests
app.use(morgan("combined"));
// Sample route
app.get("/", (req, res) => {
res.send("Hello, from Mahendra Mahara!");
});
app.listen(port, () => {
console.log(`Server is running on port ${port}`);
});
NoSQL-Based Databases
In backend development, databases play incredibly important role in storing and managing application records or data for further use or calculations. In today’s scenario, there are two primary types of databases commonly used: SQL (Structured Query Language) based databases, which fall under RDBMS, and NoSQL (Not Only SQL) databases, which use a non-structured, collection-based approach.
NoSQL databases store data in formats like JSON, which makes them easy to work with and implement. As a result, many Express.js applications use NoSQL databases, especially since their object-based nature is familiar to JavaScript developers. This compatibility makes it easier to understand and manage the database components. So, there is various database management systems handle NoSQL databases, including MongoDB, CouchDB, Redis, Amazon DynamoDB, Cassandra, Node4j, Couchbase, and more.
In our session, we focused on implementing MongoDB. Here I'll demonstrate how to connect our Express server with MongoDB, enabling us to effectively manage and utilize our application's data.
What is MongoDB?
MongoDB is a popular NoSQL database management system, much like MySQL or MariaDB in the realm of RDBMS. However, instead of storing data in tables with rows and columns, MongoDB stores data in key-value pairs using a document-oriented data model. This flexible format, similar to JSON, allows for a more natural representation of complex data structures.
MongoDB boasts features such as a flexible schema, JSON-like documents (BSON), rich querying, indexing, horizontal scalability through sharding, and a strong ecosystem with tools like MongoDB Atlas (a cloud service) and MongoDB Compass (a GUI). Its legendary feature set and strong community support have made it one of the most popular NoSQL databases in the market.
Before implementing MongoDB, it's essential to understand its core components:
+ Project: When using MongoDB Atlas (MongoDB’s cloud service), a project typically refers to a collection of databases and resources managed together. In Atlas, a project is an organizational unit containing multiple clusters and databases. It helps manage resources like clusters, user access, and billing settings under a single umbrella, making it easier to organize and control database deployments for different applications or environments (e.g., development, testing, production).
+ Cluster: A cluster is a group of MongoDB servers (or nodes) that work together to store data and provide high availability, redundancy, and horizontal scalability. In NoSQL, data normalization is not typically used, so a single collection may not be sufficient for all data needs. Using multiple databases helps reduce data duplication and minimizes dependencies on a single database. This setup also makes it easier to fix bugs and errors, as issues can be isolated to specific databases. Moreover, if one database goes down or encounters errors, the application can continue to function efficiently because other databases remain operational. Clusters manage these databases and their servers, and they can be deployed in different configurations, such as:
Clusters are essential for scaling out databases and ensuring high availability and performance. They enable MongoDB to handle large datasets and a high volume of simultaneous operations efficiently.
+ Database: In MongoDB, a database is a logical container that organizes collections. Each database contains a set of collections and has its own set of files on the file system. A single MongoDB deployment can host multiple databases, each functioning independently. This setup helps organize and separate data for different applications, teams, or purposes. Databases provide a level of isolation and can have their own permissions and settings, allowing for secure and efficient data management.
+ Collection: In MongoDB, a collection is a grouping of documents. It's similar to a table in relational databases but without a fixed schema. This means that documents within a collection can have different structures.
领英推荐
Collections organize documents into groups that are typically related in terms of application functionality or data characteristics. They provide a way to manage and query groups of documents effectively.
Steps to Connect MongoDB to an Express.js Server:
Step 1: Create a MongoDB Account
First of all, we visited the MongoDB Official Site at https://www.mongodb.com/. We signed up for a new account or logged in with our existing credentials.
Step 2: Set Up a New Cluster
Once logged in, navigate to the MongoDB Atlas dashboard. Click on "Build a Cluster" or "New Cluster" to create a new cluster. Choose your preferred cloud provider (AWS, Azure, Google Cloud) and select the region where you want to host your data.
Step 2: Set Up a New Cluster
Once logged in, we navigated to the MongoDB Atlas dashboard. We clicked on "Build a Cluster" or "New Cluster" to create a new cluster. We chose our preferred cloud provider (AWS, Azure, Google Cloud) and selected the region where we wanted to host our data. For now, we are using an energy-saving server based in Delhi, India.
Step 3: Configure Network Access
After our cluster was set up, we went to the "Security" tab and clicked on "Network Access." Here, we clicked "Add IP Address" and entered 0.0.0.0/0 to allow access from any IP address. This setting is convenient for development but not recommended for production due to security reasons.
Step 4: Add Database User
Still under the "Security" tab, we selected "Database Access." We clicked "Add New Database User," and set the username to "admin" and the password to "admin" (or chose a strong, unique password). This became our database user's credentials.
Step 5: Deployment and Connecting to the Database
We went to the "Deployment" section in the MongoDB Atlas dashboard. We clicked on our cluster and then selected "Connect." Under the "Connect" menu, we chose "Connect Your Application." We selected "Node.js" as our driver and version. We copied the provided connection string, which looked like this:
mongodb+srv://<username>:<password>@cluster0.mongodb.net/<dbname>?retryWrites=true&w=majority
Step 6: Update the Connection String
We replaced <username> with "admin" and <password> with the password we set in Step 4. We ensured that any special characters in our password were URL encoded.
Step 7: Set Up Your Express.js Project
In our development environment, we created a new directory for our project and initialized a Node.js project:
mkdir myapp
cd myapp
npm init -y
In my case, I had already done that, so there was no need to create a new project for me. We installed the necessary packages, including Express and Mongoose:
npm install express mongoose
Step 8: Connect to MongoDB in Your Server File
In our project, we created a file named “connection.js” to manage the database connection. Then, we include the connection code into the “server.js” main file. Here’s an example:
***********connection.js***********
const mongoose = require("mongoose")
mongoose.connect(process.env.DATABASE, {
useNewUrlParser:true,
useUnifiedTopology:true
}).then(console.log("MongoDB Database connected successfully")).catch((err)=>console.log("Something went wrong! ",err));
***********server.js***********
require("./database/connection");
Step 9: Start Your Server and Verify the Connection
We ran our server with “node server.js”. However, we use the “npm start” command because I have the server state management package “nodemon” already installed. If the connection was successful, we saw a message like "MongoDB Database connected successfully..." in our console, confirming that our Express.js server was now connected to our MongoDB database.
Following these steps, we set up MongoDB Atlas, configured access, and connected our database to an Express.js server using Mongoose.
Instructor: