Analysis of Node middleware Koa&Express: principle and Implementation

When it comes to middleware, many developers will think of KOA JS, among which the middleware design is undoubtedly one of the typical representatives of the front-end middleware idea.

Recently, I reviewed this part of the content again. I can't help but want to talk to you about it. It's wonderful!

Koa is very convenient to use - compared with express, its "perfect middleware" design makes the functions look very simple! The author has used this method in the project:

const Koa=require('koa')
const app=new Koa()
const Router=require('koa-router')
const router=new Router()
const cors=require('koa2-cors')
const koaBody=require('koa-body')

const ENV='test-mpin2'

app.use(cors({
	origin:['http://localhost:9528 '], / / can also be written as: [' * ']
	credentials:true
}))
app.use(koaBody({
	multipart:true
}))
app.use(async(ctx,next)=>{
	console.log('Access global Middleware')
	ctx.state.env=ENV   // Global cache
	await next()
})

const playlist=require('./controller/playlist.js')
router.use('/playlist',playlist.routes())
const blog=require('./controller/blog.js')
router.use('/blog',blog.routes())

app.use(router.routes()).use(router.allowedMethods())

app.listen(3000,()=>{
	console.log('Service turned on')
})

It pulls out the router and uses it as a separate middleware, so the app is only responsible for global processing. For example:

// The outermost middleware can be used to reveal the global error of Koa
app.use(async (ctx, next) => {
  try {
    // Execute the next Middleware
    await next();
  } catch (error) {
    console.log(`[koa error]: ${error.message}`)
  }
});
// The second layer middleware can be used for logging
app.use(async (ctx, next) => {
  const { req } = ctx;
  console.log(`req is ${JSON.stringify(req)}`);
  await next();
  console.log(`res is ${JSON.stringify(ctx.res)}`);
});

Simply implement a Koa!

In the above code, let's look at the example of Koa. The middleware is registered and connected in series through the use method. The simple implementation of its source code can be expressed as follows:

use(fn) {
    this.middleware.push(fn);
    return this;
}

We store the middleware in this In the middleware array, how is the middleware executed? Refer to the following source code:

// Start a node through the createServer method JS service
listen(...args) {
    const server = http.createServer(this.callback());
    server.listen(...args);
}

The Koa framework creates a node through the createServer method of the http module JS service and pass in this The callback () method. The callback source code is simply implemented as follows:

callback(){
	const fn=compose(this.middlewareList)
	
	return (req,res)=>{
		const ctx=createContext(req,res)
		return this.handleRequest(ctx,fn)
	}
}

handleRequest(ctx, fn) {
    const onerror = err => ctx.onerror(err);
    // Pass ctx object to middleware function fn
    return fn(ctx).catch(onerror);
}

In the above code, we comb the combination and execution process of Koa middleware into the following steps:

  1. Combine various middleware through a method (we call compose) and return a middleware combination function fn
  2. When the request comes, the handleRequest method will be called first, which completes:
    • Call createContext method to encapsulate a ctx object for this request;
    • Then call this Handlerequest (CTX, FN) handles the request.

Among them, the core process is to use the compose method to combine various Middleware - this is a separate method, which should not be constrained by other methods of Koa. Its source code is simply implemented as follows:

// Composite Middleware
// It has the same meaning as the next function in express
function compose(middlewareList){
	// return function means to return a function
	return function(ctx,next){
		// Logic of various middleware calls
		function dispatch(i){
			const fn=middlewareList[i] || next
			if(fn){
				try{
					// In koa, it is async, which returns a promise (object)
					return Promise.resolve(fn(ctx,function next(){
						return dispatch(i+1)
					}))
				}catch(err){
					return Promise.reject(err)
				}
			}else{
				return Promise.resolve()
			}
		}
		return dispatch(0)
	}
}

Its functions can be expressed as follows (non source code):

async function middleware1() {
  //...
  await (async function middleware2() {
    //...
    await (async function middleware3() {
      //...
    });
    //...
  });
  //...
}

Here we can actually "get a first glimpse" of its principle. There are two points:

  • Koa's middleware mechanism is vividly summarized by the community as an onion model;

The so-called onion model means that each Koa middleware is an onion ring, which can take charge of request entry and response return. In other words: the outer middleware can affect the request and response phase of the inner layer, and the inner middleware can only affect the response phase of the outer layer.

  • dispatch(n) corresponds to the execution of the nth middleware. In use, that is, the nth middleware can "insert" the next middleware through await next(). At the same time, it still has the ability to resume the execution after the execution of the last middleware. That is, through the onion model, await next() controls the middleware behind the call until there is no executable middleware globally and the stack is executed, and finally "return to the original path" to the middleware that executes next first. This method has an advantage, especially for global functions such as logging and error handling.

The middleware implementation of Koa1 uses the Generator function + CO Library (a Promise based Generator function process management tool) to realize the co process operation. In essence, the idea of Koa v1 middleware is similar to that of Koa v2 middleware, but Koa v2 uses Async/Await to replace the Generator function + CO library, which makes the overall implementation more ingenious and the code more elegant—— from wolf book

After the description of some of the above source codes, we can combine them in the way of es6:

// myKoa.js file

const http=require('http')

function compose(){}   //See above

class LikeKoa2{
	constructor() {
	    this.middlewareList=[]
	}
	use(){}   //See above
	
	// Give all req, res attributes and events to ctx
	createContext(req,res){
		const ctx={
			req,
			res
		}
		// such as
		ctx.query=req,query
		return ctx
	}
	handleRequest(){}   //See above
	callback(){}   //See above
	listen(){}   //See above
}

// One of the differences between koa and express:
// Express directly calls the function when calling: const app=express(); So exposed new objects -- see the code in the link below for details
// However, KOA is called in the form of class: const app=new Koa(); So expose it directly
module.exports=LikeKoa2

The use method is different from other methods. How is it implemented? After the createServer is executed, is it equivalent to establishing a channel and mounting a listening function?
I'm afraid this will be explored in the source code of Node

Compare with Koa and talk about the principle of Express

Speaking of node JS framework, we must not forget Express - unlike Koa, it inherits the functions of routing, static server and template engine. Although it is much "bloated" than Koa, it looks more like a framework than Koa. By learning the express source code, the author briefly summarizes its working mechanism:

  1. Via app Use method to register middleware.
  2. A middleware can be understood as a Layer object, which contains the regular information of the current route matching and the handle method.
  3. All middleware (Layer objects) are stored in stack array.
  4. When a request comes, it will get the request path from req and find the matching Layer from the stack according to the path. The specific matching process is determined by router Handle function implementation.
  5. router. The handle function traverses each layer through the next() method for comparison:
    • The next() method maintains the reference to the Stack Index cursor through the closure. When the next() method is called, it will start from the next middleware;
    • If the comparison result is true, call layer handle_ Request method, layer handle_ The next() method will be called in the request method to implement the execution of the middleware.

From the above, we can see that Express actually maintains the Index cursor traversing the middleware list through the next() method. Each time the middleware calls the next() method, it will find and execute the next middleware by adding the Index cursor. Its function is like this:

((req, res) => {
  console.log('First Middleware');
  ((req, res) => {
    console.log('Second Middleware');
    (async(req, res) => {
      console.log('The third Middleware');
      await sleep(2000)
      res.status(200).send('hello')
    })(req, res)
    console.log('The second middleware call ends');
  })(req, res)
  console.log('The first middleware call ends')
})(req, res)

According to the above code, the design of Express middleware is not an onion model. It is a linear model based on callback implementation, which is not conducive to combination and interoperability. The design is not as simple as Koa. Moreover, the business code has a certain degree of intrusion, and even causes the coupling between different middleware.

The simple implementation of express has been uploaded to Tencent micro cloud by the author. Those who need it can view & Download: Simple implementation of express

Posted by khalidorama on Tue, 19 Apr 2022 03:20:07 +0930