Node JS Architecture

The credit goes to Original Author - ClickHere

Node JS Architecture

Before starting some Node JS programming examples, it’s important to have an idea about Node JS architecture. We will discuss about “How Node JS works under-the-hood, what type of processing model it is following, How Node JS handles concurrent request with Single-Threaded model” etc. in this post.

Node JS Single Threaded Event Loop Model

As we have already discussed, Node JS applications uses “Single Threaded Event Loop Model” architecture to handle multiple concurrent clients.

There are many web application technologies like JSP, Spring MVC, ASP.NET, HTML, Ajax, jQuery etc. But all these technologies follow “Multi-Threaded Request-Response” architecture to handle multiple concurrent clients.

We are already familiar with “Multi-Threaded Request-Response” architecture because it’s used by most of the web application frameworks. But why Node JS Platform has chosen different architecture to develop web applications. What is the major differences between multithreaded and single threaded event loop architecture.

Any web developer can learn Node JS and develop applications very easily. However without understanding Node JS Internals, we cannot design and develop Node JS Applications very well. So before starting developing Node JS Applications, first we will learn Node JS Platform internals.

Node JS Platform

Node JS Platform uses “Single Threaded Event Loop” architecture to handle multiple concurrent clients. Then how it really handles concurrent client requests without using multiple threads. What is Event Loop model? We will discuss these concepts one by one.

Before discussing “Single Threaded Event Loop” architecture, first we will go through famous “Multi-Threaded Request-Response” architecture.

Traditional Web Application Processing Model

Any Web Application developed without Node JS, typically follows “Multi-Threaded Request-Response” model. Simply we can call this model as Request/Response Model.

Client sends request to the server, then server do some processing based on clients request, prepare response and send it back to the client.

This model uses HTTP protocol. As HTTP is a Stateless Protocol, this Request/Response model is also Stateless Model. So we can call this as Request/Response Stateless Model.

However, this model uses Multiple Threads to handle concurrent client requests. Before discussing this model internals, first go through the diagram below.

Request/Response Model Processing Steps:

• Clients Send request to Web Server.
• Web Server internally maintains a Limited Thread pool to provide services to the Client Requests.
• Web Server is in infinite Loop and waiting for Client Incoming Requests
• Web Server receives those requests.
  • Web Server pickup one Client Request
  • Pickup one Thread from Thread pool
  • Assign this Thread to Client Request
  • This Thread will take care of reading Client request, processing Client request, performing any Blocking IO Operations (if required) and preparing Response
  • This Thread sends prepared response back to the Web Server
  • Web Server in-turn sends this response to the respective Client.

Server waits in Infinite loop and performs all sub-steps as mentioned above for all n clients. That means this model creates one Thread per Client request.

If more clients requests require Blocking IO Operations, then almost all threads are busy in preparing their responses. Then remaining clients Requests should wait for longer time.

Diagram Description:

• Here “n” number of Clients Send request to Web Server. Let us assume they are accessing our Web Application concurrently.
• Let us assume, our Clients are Client-1, Client-2… and Client-n.
• Web Server internally maintains a Limited Thread pool. Let us assume “m” number of Threads in Thread pool.
• Web Server receives those requests one by one.
  • Web Server pickup Client-1 Request-1, Pickup one Thread T-1 from Thread pool and assign this request to Thread T-1
    • Thread T-1 reads Client-1 Request-1 and process it
    • Client-1 Request-1 does not require any Blocking IO Operations
    • Thread T-1 does necessary steps and prepares Response-1 and send it back to the Server
    • Web Server in-turn send this Response-1 to the Client-1
  • Web Server pickup another Client-2 Request-2, Pickup one Thread T-2 from Thread pool and assign this request to Thread T-2
    • Thread T-2 reads Client-1 Request-2 and process it
    • Client-1 Request-2 does not require any Blocking IO Operations
    • Thread T-2 does necessary steps and prepares Response-2 and send it back to the Server
    • Web Server in-turn send this Response-2 to the Client-2
  • Web Server pickup another Client-n Request-n, Pickup one Thread T-n from Thread pool and assign this request to Thread T-n
    • Thread T-n reads Client-n Request-n and process it
    • Client-n Request-n require heavy Blocking IO and computation Operations
    • Thread T-n takes more time to interact with external systems, does necessary steps and prepares Response-n and send it back to the Server
    • Web Server in-turn send this Response-n to the Client-n
  If “n” is greater than “m” (Most of the times, its true), then server assigns Threads to Client Requests up to available Threads. After all m Threads are utilized, then remaining Client’s Request should wait in the Queue until some of the busy Threads finish their Request-Processing Job and free to pick up next Request.
  If those threads are busy with Blocking IO Tasks (For example, interacting with Database, file system, JMS Queue, external services etc.) for longer time, then remaining clients should wait longer time.
• Once Threads are free in Thread Pool and available for next tasks, Server pickup those threads and assign them to remaining Client Requests.
• Each Thread utilizes many resources like memory etc. So before going those Threads from busy state to waiting state, they should release all acquired resources.

Drawbacks of Request/Response Stateless Model:

• Handling more and more concurrent client’s request is bit tough.
• When Concurrent client requests increases, then it should use more and more threads, finally they eat up more memory.
• Sometimes, Client’s Request should wait for available threads to process their requests.
• Wastes time in processing Blocking IO Tasks.

Node JS Architecture – Single Threaded Event Loop

Node JS Platform does not follow Request/Response Multi-Threaded Stateless Model. It follows Single Threaded with Event Loop Model. Node JS Processing model mainly based on Javascript Event based model with Javascript callback mechanism.

You should have some good knowledge about how Javascript events and callback mechanism works. If you don’t know, Please go through those posts or tutorials first and get some idea before moving to the next step in this post.

As Node JS follows this architecture, it can handle more and more concurrent client requests very easily. Before discussing this model internals, first go through the diagram below.

I tried to design this diagram to explain each and every point of Node JS Internals.

The main heart of Node JS Processing model is “Event Loop”. If we understand this, then it is very easy to understand the Node JS Internals.

Single Threaded Event Loop Model Processing Steps:

• Clients Send request to Web Server.
• Node JS Web Server internally maintains a Limited Thread pool to provide services to the Client Requests.
• Node JS Web Server receives those requests and places them into a Queue. It is known as “Event Queue”.
• Node JS Web Server internally has a Component, known as “Event Loop”. Why it got this name is that it uses indefinite loop to receive requests and process them. (See some Java Pseudo code to understand this below).
• Event Loop uses Single Thread only. It is main heart of Node JS Platform Processing Model.
• Even Loop checks any Client Request is placed in Event Queue. If no, then wait for incoming requests for indefinitely.
• If yes, then pick up one Client Request from Event Queue
  • Starts process that Client Request
  • If that Client Request Does Not requires any Blocking IO Operations, then process everything, prepare response and send it back to client.
  • If that Client Request requires some Blocking IO Operations like interacting with Database, File System, External Services then it will follow different approach
    • Checks Threads availability from Internal Thread Pool
    • Picks up one Thread and assign this Client Request to that thread.
    • That Thread is responsible for taking that request, process it, perform Blocking IO operations, prepare response and send it back to the Event Loop
    • Event Loop in turn, sends that Response to the respective Client.

Diagram Description:

• Here “n” number of Clients Send request to Web Server. Let us assume they are accessing our Web Application concurrently.
• Let us assume, our Clients are Client-1, Client-2… and Client-n.
• Web Server internally maintains a Limited Thread pool. Let us assume “m” number of Threads in Thread pool.
• Node JS Web Server receives Client-1, Client-2… and Client-n Requests and places them in the Event Queue.
• Node JS Even Loop Picks up those requests one by one.
  • Even Loop pickups Client-1 Request-1
    • Checks whether Client-1 Request-1 does require any Blocking IO Operations or takes more time for complex computation tasks.
    • As this request is simple computation and Non-Blocking IO task, it does not require separate Thread to process it.
    • Event Loop process all steps provided in that Client-1 Request-1 Operation (Here Operations means Java Script’s functions) and prepares Response-1
    • Event Loop sends Response-1 to Client-1
  • Even Loop pickups Client-2 Request-2
    • Checks whether Client-2 Request-2does require any Blocking IO Operations or takes more time for complex computation tasks.
    • As this request is simple computation and Non-Blocking IO task, it does not require separate Thread to process it.
    • Event Loop process all steps provided in that Client-2 Request-2 Operation and prepares Response-2
    • Event Loop sends Response-2 to Client-2
  • Even Loop pickups Client-n Request-n
    • Checks whether Client-n Request-n does require any Blocking IO Operations or takes more time for complex computation tasks.
    • As this request is very complex computation or Blocking IO task, Even Loop does not process this request.
    • Event Loop picks up Thread T-1 from Internal Thread pool and assigns this Client-n Request-n to Thread T-1
    • Thread T-1 reads and process Request-n, perform necessary Blocking IO or Computation task, and finally prepares Response-n
    • Thread T-1 sends this Response-n to Event Loop
    • Event Loop in turn, sends this Response-n to Client-n

Here Client Request is a call to one or more Java Script Functions. Java Script Functions may call other functions or may utilize its Callback functions nature.

So Each Client Request looks like as shown below:

For Example:

function1(function2,callback1);
function2(function3,callback2);
function3(input-params);

NOTE: –

• If you don’t understand how these functions are executed, then I feel you are not familiar with Java Script Functions and Callback mechanism.
• We should have some idea about Java Script functions and Callback mechanisms. Please go through some online tutorial before starting our Node JS Application development.

Node JS Architecture – Single Threaded Event Loop Advantages

1. Handling more and more concurrent client’s request is very easy.
2. Even though our Node JS Application receives more and more Concurrent client requests, there is no need of creating more and more threads, because of Event loop.
3. Node JS application uses less Threads so that it can utilize only less resources or memory

Event Loop Pseudo Code

As I’m a Java Developer, I will try to explain “How Event Loop works” in Java terminology. It is not in pure Java code, I guess everyone can understand this. If you face any issues in understanding this, please drop me a comment.

public class EventLoop {
while(true){
         if(Event Queue receives a JavaScript Function Call){
          ClientRequest request = EventQueue.getClientRequest();
                            If(request requires BlokingIO or takes more computation time)
                                    Assign request to Thread T1
                            Else
                                  Process and Prepare response
                  }
            }
} 

That’s all for Node JS Architecture and Node JS single threaded event loop.

Node JS Must DO in Production Tips

**CreditgGoes to actual Author here - Click Here

At Hashnode we heavily use Node.js. I am a big fan of it and have learned a lot of things while running Hashnode. When I hang out with other developers, I notice that many people don't utilise Node to its full potential and do certain things the wrong way. So, this article is going to be about things which you shouldn't do while running Node in production. Let's get started!

Not using Node.js Cluster

Node.js is single threaded in nature and has a memory limit of 1.5GB. Due to this, it can't take advantage of multiple CPU cores automatically. But the good news is that Cluster module lets you fork multiple child processes which will use IPC to communicate with the parent process. The master process controls the workers and all the incoming connections are distributed across the workers in a round robin fashion.

Clustering improves your app's performance and lets you achieve zero downtime (hot) deployments easily (More on this later). Also keep in mind that number of workers that can be created is not limited by the number of CPU cores of the machine.

I feel clustering is a must-have for any production Node.js app and there is no reason not to use it.

Performing heavy lifting inside web servers

Node/Express servers aren't meant to perform heavy and computationally intensive tasks. For instance, in a typical web app you will have to send bulk emails to users. Although you can perform this task in the Node.js web server itself, it will degrade the performance significantly. It is always better to break these heavy tasks into micro services and deploy them as separate Node apps. Further you can use a message queue like RabbitMQ to communicate with these micro services.

When you post something on Hashnode and tag it with various nodes, we insert it into the feeds of thousands of users. This is a heavy task. Until a few months back, the feed insertion task was being handled by the web servers themselves. As the traffic increased, we started to see performance bottlenecks. Back then if you would tag a question with "General Programming" or "JavaScript" and post it, the whole website would become non-responsive for a few seconds. Proper investigation revealed that it's due to the feed insertion activity. The solution was to move that particular module into a different machine and initiate the task via a message queue.

So, the key take away is that Node.js is best suited for event handling and non blocking I/O. Any task that would take long to complete should be handled by a separate process.

Not using a process manager

While it's obvious that process managers have a lot to offer, many first time Node users deploy their apps to production without a process manager. At Hashnode we have been using pm2, a powerful process manager for Node.js.

Also, if you use pm2 you can start your app in cluster mode very easily.

pm2 start app.js -i 2

In the above example, i specifies the number of workers you want to run in cluster mode. The best part is that you can now reload the workers one after another so that your app doesn't suffer any downtime during deployment. The following command does it :

pm2 reload app

If you happen to use pm2, do check out Keymetrics which is a monitoring service for Node.js (based on pm2).

Not using a reverse proxy

I have seen developers running Node based apps on port 80 and serving static files through it. You should remember that running a Node app on port 80 is not a good idea and is dangerous in most cases. Instead you should run the app on a different port like 3000 and use nginx (or something like HAProxy?) as a reverse proxy in front of the Node.js app.

The above setup protects your application servers from direct exposure to internet traffic and helps you scale the servers and load balance them easily.

Lack of monitoring

Bad things like unexpected errors, exceptions will keep happening all the time. You know what's worse? It's not knowing that something bad happened in your Node process. Now that you are using a process manager, your node process will be reloaded whenever an unhandled exception occurs. So, unless you check the logs you won't find out the issue. The solution is to use a monitoring service and have them alert you via email/sms in case your process gets killed and restarted.

Not removing console.log statements

While developing an app, we use console.log statements to test things out. But sometimes we forget to remove these log statements in production, which consume the CPU time and waste the resources. The best way to avoid this is to use debug module. So, unless you start your app with environment variable DEBUG nothing will be printed to the console.

Maintaining global states inside the Node web processes

Sometimes developers store things like session ids, socket connections etc inside the memory. This is a big NO and should be avoided at all cost. If you do store session ids in memory, you will see that your users are logged out as soon as you restart the server. This also causes problems while scaling the app and adding more servers. Your web servers should just handle web traffic and should not maintain any kind of state in memory.

Not using SSL

For a user facing website, there is no reason not to force SSL by default. Sometimes I also see developers reading SSL keys from a file and using them in the Node process. You should always use a reverse proxy in front of your Node.js app and install SSL on that.

Also, keep checking for latest SSL vulnerabilities and apply fixes ASAP.

Lack of basic security measures

Security is always important, and it's good to be paranoid about your app's security. In addition to the basic security checks, you should use something like NSP to discover vulnerabilities in your project.

Also, don't use outdated versions of Node and Express in production. They are no longer maintained and don't receive security updates.

Not using a VPN

Always deploy your app inside a private network, so that only trusted clients can communicate with your servers. Often while deploying, people forget this simple thing and face a lot of problems later. It's always a good practice to think of the infrastructure and architecture in advance, before deploying your app.

For instance, if your Node server runs on port 8080 and you have setup nginx as a reverse proxy, it's important to make sure that only nginx can connect to your app on the specific port. It should be isolated from the rest of the world.

---

More:

• Serve static assets not minified and gzip'ed (see grunt)
• Disallow local caching for those static assets
• Keep access logging enabled
• Store uploaded files on disk folders instead of a virtual file system like GridFS

Discussion:

Not using a reverse proxy

Most reverse proxies are not able to fully leverage the power of HTTP/2 and other modern standards. Sometimes, there is trouble with reverse-proxies and (web-)socket connections. I think there are cases where you really want to expose your Node.JS application to the internet.

Maintaining global states inside Node web processes

How to maintain state really depends on the kind of application you develop. I work on an application which keeps alive websocket connections with some 500 devices. They have to be manageable over a separate GUI, so I do not have any other choice than to store the connections, if I want to reuse them (and send commands) based on the device-name.

Comments:

yeah, that might be one solution, but why not just write everything you need directly in Node.JS? Even with Go, the service will not perform any better as Node.JS will still be the slowest part in the chain

Using Cluster:

i'm building a node app which is completely stateless and uses JWT for authentication. i'm planning to host the app on 10 servers each having only i cpu CORE (as it is stateless there is no need to bother about session sync) each server has nginx to frontload node app on that particular server . and all these servers will be front loaded by 1 load balancer . so my question is in my scenario do i really require cluster module ?

Comments:

even having only 1 CPU core, the cluster stills brings to you the benefit of having no downtime when reloading the application on a specific server. Yeah, I know... you already have other 9 servers to take care when 1 server is restarting the app... It may be a "micro-benefit", but it is still a benefit. (Specially if you start with one server alone before adding the others.)

Reference:

https://nodejs.org/api/cluster.html#cluster_how_it_works

Node JS Fundamentals

Node Architecture

Event Loop