High performance Java Communication is absolutely inseparable from Java NIO technology. Now mainstream technical frameworks or middleware servers use Java NIO technology, such as Tomcat, Jetty and Netty. Learning and mastering NIO technology is not a bonus skill, but a necessary skill. Whether it's interview or actual development, as the "siege lion" of Java (homonym of engineer), you must master the principle and development practice skills of NIO.
Introduction to Java NIO
Before version 1.4, the Java IO class library was blocked IO; Since version 1.4, a new asynchronous IO library has been introduced, which is called Java New IO class library, or JAVA NIO for short.
The goal of the New IO class library is to make Java support non blocking io. For this reason, more people like to call Java NIO non block IO and "old" blocking Java IO OIO (Old IO).
Generally speaking, NIO makes up for the deficiency of the original flow oriented OIO synchronization blocking. It provides high-speed and buffer oriented IO for standard Java code.
Java NIO consists of the following three core components: · Channel · Buffer · Selector if you understand the four IO models in Chapter 1, you can recognize at a glance that JavaNIO belongs to the third model - IO multiplexing model. Of course, Java NIO components provide a unified API to shield the differences between different operating systems at the bottom. In the following chapters, we will introduce the above three core components of Java NIO in detail. Let's take a look at the simple comparison between NIO and OIO in Java.
Comparison between NIO and OIO
In Java, the difference between NIO and OIO is mainly reflected in three aspects:
(1) OIO is Stream Oriented and NIO is Buffer Oriented. What is Stream Oriented and Buffer Oriented? OIO is oriented to byte Stream or character Stream. In general OIO operation, we sequentially read one or more bytes from a Stream in a streaming manner. Therefore, we cannot change the position of the read pointer at will. It is different in NIO operation. NIO introduces the concepts of Channel and Buffer. For reading and writing, you only need to read data from the Channel to the Buffer, or write data from the Buffer to the Channel. NIO is not a sequential operation like OIO. You can read data anywhere in the Buffer at will.
(2) The operation of OIO is blocking, while the operation of NIO is non blocking. How can NIO be non blocking? As we all know, OIO operations are blocked. For example, if we call a read method to read the contents of a file, the thread calling read will be blocked until the read operation is completed. In the non blocking mode of NIO, when we call the read method, if there is data at this time, read reads the data and returns; If there is no data at this time, read returns directly without blocking the current thread. How does NIO achieve non blocking? In fact, in the last chapter, the answer has been revealed. NIO uses channel and channel multiplexing technology.
(3) OIO has no concept of Selector, while NIO has the concept of Selector. The implementation of NIO is based on the system call of the underlying Selector. NIO's Selector needs the support of the underlying operating system. OIO does not need a Selector.
Channel
In OIO, the same network connection is associated with two streams: an input stream and an Output Stream. Through these two streams, the input and output operations are carried out continuously.
In NIO, the same network connection is represented by a channel. All IO operations of NIO start from the channel. A channel is similar to the combination of two streams in OIO. It can be read from or written to the channel.
Selector selector
First, review a basic question, what is IO multiplexing?
It means that a process / thread can monitor multiple file descriptors at the same time (a network connection is represented by a file descriptor at the bottom of the operating system). Once one or more file descriptors are readable or writable, the system kernel will notify the process / thread.
At the Java application level, how to monitor multiple file descriptors? You need to use a very important Java NIO component - the Selector selector. What's the magic of selectors? It is a query for IO events. Through the Selector, a thread can query the ready status of IO events of multiple channels.
To realize IO multiplexing, from the specific development level, first register the channels in the Selector, and then through the internal mechanism of the Selector, you can query (select) whether these registered channels have ready IO events (such as readable, writable, network connection completion, etc.). A Selector only needs one thread to monitor. In other words, we can simply use one thread to manage multiple channels through the Selector. This is very efficient. This efficiency comes from the Selector component of Java and the underlying IO multiplexing support of the operating system.
Compared with OIO, the biggest advantage of using selector is that the system overhead is small, and the system does not have to create processes / threads for each network connection (file descriptor), which greatly reduces the system overhead.
Buffer
The main interaction between the application and the Channel is to read and write data.
In order to accomplish such a big task, NIO has prepared the third important component - NIO Buffer.
Channel reading is to read data from the channel into the buffer; Channel writing is to write data from the buffer to the channel. The use of buffer is not available in flow oriented OIO, but also one of the important prerequisites and foundations of NIO non blocking.
Starting from the Buffer, we will introduce the three core components of NIO: Buffer, Channel and Selector in detail.
Explain NIO Buffer class and its attributes in detail
NIO Buffer is essentially a memory block, which can write data or read data from it.
NIO's Buffer class is an abstract class located in Java NIO package, inside which is a memory block (array). NIO Buffer is different from ordinary memory blocks (Java arrays): NIO Buffer object provides a set of more effective methods for alternate access of write and read. It should be emphasized that the Buffer class is a non thread safe class.
Buffer class
Buffer class is an abstract class, corresponding to the main data types of Java. There are eight buffer classes in NIO, which are as follows: ByteBuffer, CharBuffer, DoubleBuffer, FloatBuffer, IntBuffer, LongBuffer, ShortBuffer and MappedByteBuffer.
The first seven Buffer types cover all Java basic data types that can be transmitted in IO. The eighth type MappedByteBuffer is a ByteBuffer type specifically used for memory mapping. In fact, ByteBuffer binary byte Buffer type is used most, as you will see later.
Important attributes of Buffer class
Inside the Buffer class, there is a byte [] array memory block as the memory Buffer.
In order to record the status and location of reading and writing, the Buffer class provides some important properties. There are three important member attributes: capacity, position and limit. In addition, there is a tag attribute: mark, which can temporarily store the current position in mark; If necessary, you can recover from the mark mark to the position position.
capacity attribute
The capacity attribute of Buffer class indicates the internal capacity. Once the number of objects written exceeds the capacity, the Buffer is full and can no longer be written.
Once the capacity attribute of Buffer class is initialized, it cannot be changed. Why? During initialization of Buffer class objects, internal memory will be allocated according to capacity. After the memory is allocated, of course, its size cannot be changed.
Again, capacity does not refer to the number of bytes in the byte [] array of memory blocks. Capacity refers to the number of data objects written.
As mentioned earlier, Buffer class is an abstract class, and Java cannot be directly used to create new objects. When using, you must use a subclass of Buffer. For example, if you use DoubleBuffer, the data written is of double type. If its capacity is 100, we can write up to 100 double data.
position attribute
The position attribute of Buffer class indicates the current position. The position attribute is related to the read-write mode of the Buffer. In different modes, the value of position attribute is different. When the mode of reading and writing in the Buffer changes, the position will be adjusted.
In write mode, the value change rule of position is as follows:
(1) When you first enter the write mode, the position value is 0, indicating that the current write position is from scratch.
(2) Every time a data is written to the buffer, position moves backward to the next writable location.
(3) The initial position value is 0, and the maximum writable position value is limit – 1. When the position value reaches the limit, the buffer has no space to write.
In the read mode, the value change rule of position is as follows:
(1) When the buffer first enters read mode, position is reset to 0.
(2) When reading from the buffer, it also starts from the position position. After reading the data, position moves forward to the next readable position.
(3) The maximum value of position is the maximum readable upper limit. When position reaches the limit, it indicates that there is no data readable in the buffer. Where is the starting point? When a new buffer is created, the buffer is in write mode, and data can be written at this time.
After data is written, if you want to read data from the buffer, you need to switch the mode. You can use (call) the flip flip method to change the buffer into the read mode. During the flip flip process, the position will be greatly adjusted. The specific rule is: the position changes from the original write position to a new readable position, that is, 0, which means it can be read from scratch. The other half of flip flip is to adjust the limit attribute.
limit attribute
The limit attribute of Buffer class indicates the maximum upper limit of reading and writing.
The limit attribute is also related to the read-write mode of the buffer. In different modes, the meaning of limit value is different.
In write mode, the meaning of limit attribute value is the maximum upper limit of data that can be written. When you first enter the write mode, the limit value will be set to the capacity value of the buffer, indicating that you can always write the buffer full. In read mode, the value of limit means how much data can be read from the buffer at most.
Generally speaking, write before read. After the Buffer is written, you can start reading data from the Buffer. You can use the flip flip method. At this time, the value of limit will also be greatly adjusted.
How to adjust it? Set the position value in write mode to the limit value in read mode, that is, take the maximum number of previous writes as the upper limit value that can be read. During flip flip flip, the adjustment of attributes will involve two attributes: position and limit. This adjustment is subtle and not easy to understand,
A simple example: first, create a buffer. At first, the buffer is in write mode. Position is 0 and limit is the maximum capacity. Then, write data to the buffer. Every time a data is written, position moves one position to the back, that is, the value of position plus 1. It is assumed that five numbers are written. When the writing is completed, the value of position is 5. At this time, use (that is, call) the flip method to switch the buffer to read mode. The value of limit will first be set to the position value in write mode. Here, the new limit is 5, which means that the maximum number that can be read is 5. At the same time, the new position will be reset to 0, indicating that it can be read from 0.
Summary of 4 attributes
In addition to the previous three attributes, the fourth attribute mark is relatively simple. It is equivalent to a temporary attribute, which temporarily saves the value of position to facilitate the reuse of position value later. The following table summarizes the four important attributes of Buffer class. See table 3-1. Table 3-1 value description of four important attributes of Buffer
Explain the important methods of NIO Buffer class in detail
This section will introduce several methods commonly used in the use of Buffer class in detail, including obtaining Buffer instance, writing, reading, repeated reading, marking and resetting Buffer instance, etc.
allocate()
Create Buffer before using Buffer, we first need to obtain the instance object of Buffer subclass and allocate memory space.
In order to obtain a Buffer instance object, instead of using the subclass constructor new to create an instance object, we call the subclass's allocate() method. The following program fragment is used to obtain the Buffer instance object of an integer Buffer class. The code is as follows:
package com.crazymakercircle.bufferDemo;
//...
public class UseBuffer {
static IntBuffer intBuffer = null;
public static void allocatTest(){
//Call the allocate method instead of using new
intBuffer = IntBuffer.allocate(20);
//Output the main attribute values of buffer
Logger.info("------------after allocate-----------------");
Logger.info("position=" + intBuffer.position());
Logger.info("limit=" + intBuffer.limit());
Logger.info("capacity=" + intBuffer.capacity());
}
//
...
}
In the example, intbuffer is a specific subclass of Buffer. You can call intbuffer Allocate (20), create an intbuffer instance object, and allocate 20 * 4 bytes of memory space. Through the output result of the program, we can view the main attribute values of a new Buffer instance object, as shown below:
allocatTest |> ------------after allocate------------------ allocatTest |> position=0 allocatTest |> limit=20 allocatTest |> capacity=20
From the above operation results, we can see that after a buffer is created, it is in the write mode. The position write position is 0, the maximum writable upper limit is the initialization value (here it is 20), and the capacity of the buffer is also the initialization value.
put()
Write to the buffer. After calling the allocate method to allocate memory and returning the instance object, the buffer instance object is in write mode and can be written to the object. To write to the buffer, you need to call the put method.
The put method is very simple, with only one parameter, that is, the object to be written. However, the type of data written is required to be consistent with the type of buffer. Next, in the previous example, write five integers to the intBuffer cache instance object just created. The code is as follows:
package com.crazymakercircle.bufferDemo;
//...
public class UseBuffer {
static IntBufferintBuffer = null;
//The code for creating buffer is omitted. See the source code project for details
public static void putTest() {
for (int i = 0; i< 5; ){
//Write an integer to the buffer
intBuffer.put(i);
}
//The primary attribute value of the output buffer
Logger.info("------------after put------------------");
Logger.info("position=" + intBuffer.position());
Logger.info("limit=" + intBuffer.limit());
Logger.info("capacity=" + intBuffer.capacity());
}
//...
}
After five elements are written, the main attribute values of the buffer are also output. The output results are as follows:
putTest |> ------------after putTest------------------ putTest |> position=5 putTest |> limit=20 putTest |> capacity=20
As can be seen from the result, position becomes 5, pointing to the sixth element position that can be written. The upper limit of the maximum write element and the maximum capacity value of capacity have not changed.
lip() flip
After writing data to the buffer, can I read data directly from the buffer? Oh, No. At this time, the buffer is still in write mode. If you need to read data, you also need to convert the buffer to read mode.
flip() flip method is an important method of mode transition provided by Buffer class. Its function is to flip the write mode into the read mode.
Following the previous example, demonstrate the use of the flip() method:
package com.crazymakercircle.bufferDemo;
//...
public class UseBuffer {
static IntBufferintBuffer = null;
//The code for creating and writing the buffer is omitted. See the source code project for details
public static void flipTest() {
//Flip the buffer from write mode to read mode
intBuffer.flip();
//The primary attribute value of the output buffer
Logger.info("------------after flip ------------------");
Logger.info("position=" + intBuffer.position());
Logger.info("limit=" + intBuffer.limit());
Logger.info("capacity=" + intBuffer.capacity());
}
//...
}
After calling flip to flip the mode, the properties of the buffer have changed wonderfully. The output is as follows:
flipTest |> ------------after flipTest ------------------ flipTest |> position=0 flipTest |> limit=5 flipTest |> capacity=20
After the flip method is called, the position value 5 in the previous write mode becomes the upper readable limit value 5; The position value in the new reading mode simply and rudely changes to 0, indicating that it is read from scratch.
The rules of write to read conversion of flip() method are described in detail as follows:
First, set the upper limit of readable length. The position value of the last write position of the contents in the buffer in write mode is used as the upper limit value in read mode.
Secondly, set the value of position at the start of reading to 0, which means reading from scratch.
Finally, clear the previous mark mark, because mark saves the temporary position in write mode. In read mode, if you continue to use the old mark mark, it will cause confusion in position.
For the above three steps, you can actually check the source code of the flip method,
Buffer. The source code of the flip () method is as follows:
public final Buffer flip() {
limit = position; //Set the upper limit of the readable length to the written position
position = 0; //Set the value of position at the start of reading to 0, which means reading from scratch
mark = UNSET_MARK; // Clear previous mark mark
return this;
}
So far, we all know how to switch the buffer to read mode. A new problem comes. How to switch the buffer to write mode again after reading? You can call buffer Clear() clear or buffer Compact () compression methods, which can convert the buffer to write mode.
The mode conversion of Buffer is roughly shown in Figure 3-1.
get()
Read from the buffer and call the flip method to switch the buffer to read mode. At this point, you can start reading data from the buffer.
Reading data is very simple. Call the get method to read data from the position one at a time, and adjust the buffer properties accordingly.
Next, use the example of flip above to demonstrate the operation of reading the buffer. The code is as follows:
public static void getTest()
{
for (int i = 0; i < 2; i++)
{
int j = intBuffer.get();
Logger.debug("j = " + j);
}
Logger.debug("------------after get 2 int ------------------");
Logger.debug("position=" + intBuffer.position());
Logger.debug("limit=" + intBuffer.limit());
Logger.debug("capacity=" + intBuffer.capacity());
for (int i = 0; i < 3; i++)
{
int j = intBuffer.get();
Logger.debug("j = " + j);
}
Logger.debug("------------after get 3 int ------------------");
Logger.debug("position=" + intBuffer.position());
Logger.debug("limit=" + intBuffer.limit());
Logger.debug("capacity=" + intBuffer.capacity());
}
Read 2 first and then 3. After running, the output results are as follows:
getTest |> ------------after get 2 int ------------------ getTest |> position=2 getTest |> limit=5 getTest |> capacity=20 getTest |> ------------after get 3 int ------------------ getTest |> position=5 getTest |> limit=5 getTest |> capacity=20
From the output of the program, we can see that the read operation will change the value of the readable position, while the limit value will not change.
If the value of position is equal to the value of limit, it means that all data reading is completed. Position points to an element position without data and can no longer be read. When reading again, a BufferUnderflowException exception will be thrown.
It is emphasized here that after reading, can the write mode be carried out immediately? No. It is still in read mode, so we must call buffer Clear() or buffer Compact(), that is, empty or compress the buffer before it can be changed into write mode and make it writable again.
In addition, there is another question: can the buffer be read repeatedly? The answer is yes.
rewind() rewind
If you need to read the read data again, you can call the rewind() method.
rewind() is also called rewind, which is like playing a tape. Rewind and replay. Following the previous code, continue the demonstration of the rewind method. The example code is as follows:
public static void rewindTest()
{
intBuffer.rewind();
Logger.debug("------------after rewind ------------------");
Logger.debug("position=" + intBuffer.position());
Logger.debug("limit=" + intBuffer.limit());
Logger.debug("capacity=" + intBuffer.capacity());
}
The execution results of this example program are as follows:
rewindTest |> ------------after rewind ------------------ rewindTest |> position=0 rewindTest |> limit=5 rewindTest |> capacity=20
The rewind() method mainly adjusts the position attribute of the buffer. The specific adjustment rules are as follows:
(1) position is reset to 0, so you can reread all data in the buffer.
(2) The limit remains unchanged, and the amount of data is the same. It still indicates how many elements can be read from the buffer.
(3) The mark mark is cleared to indicate that the previous temporary location can no longer be used.
Buffer. The source code of rewind() method is as follows:
public final Buffer rewind() {
position = 0;//Reset to 0, so you can reread all data in the buffer
mark = -1; // The mark mark is cleared to indicate that the previous temporary location can no longer be used
return this;
}
Through the source code, we can see that the rewind() method is very similar to flip(). The difference is that rewind() does not affect the limit attribute value; flip() resets the limit attribute value.
After rewind, you can read it again. The example code of repeated reading is as follows:
/**
* rewind After that, repeat the reading
* And demonstrate the mark marking method
*/
public static void reRead()
{
for (int i = 0; i < 5; i++)
{
if (i == 2)
{
intBuffer.mark();
}
int j = intBuffer.get();
Logger.debug("j = " + j);
}
Logger.debug("------------after reRead------------------");
Logger.debug("position=" + intBuffer.position());
Logger.debug("limit=" + intBuffer.limit());
Logger.debug("capacity=" + intBuffer.capacity());
}
This code is basically the same as the previous reading example code, except that a mark call is added.
mark() and reset()
Buffer. The function of the mark () method is to save the value of the current position and put it in the mark attribute, so that the mark attribute can remember the temporary position;
After that, you can call buffer The reset () method restores the value of mark to position. That is, buffer Mark() and buffer The reset () method is used together. Both methods need the support of internal mark attribute.
In the previous example code of repeatedly reading the buffer, when the third element is read (i==2), call the mark() method to save the value of the current position in the mark attribute. At this time, the value of the mark attribute is 2.
Next, you can call the reset method to restore the value of the mark attribute to position. You can then start reading from position 2 (the third element).
Continue with the previous repeatedly read code to demonstrate the example of reset. The code is as follows:
public static void afterReset()
{
Logger.debug("------------after reset------------------");
//Restore the value previously saved in mark to position
intBuffer.reset();
//Attribute value of output buffer
Logger.debug("position=" + intBuffer.position());
Logger.debug("limit=" + intBuffer.limit());
Logger.debug("capacity=" + intBuffer.capacity());
for (int i = 2; i < 5; i++)
{
int j = intBuffer.get();
Logger.debug("j = " + j);
}
}
In the above code, first call reset() to restore the value in mark to position, so the read position is 2, indicating that you can start reading data from the third element again.
The output result of the above program code is:
afterReset |> ------------after reset------------------ afterReset |> position=2 afterReset |> limit=5 afterReset |> capacity=20 afterReset |> j = 2 afterReset |> j = 3 afterReset |> j = 4
After the reset method is called, the value of position is 2. At this time, read the buffer, and the three elements after the output are 2, 3 and 4.
clear() clear buffer
In read mode, call the clear() method to switch the buffer to write mode. This method will clear position and set limit to the maximum capacity value of capacity. It can be written until the buffer is full. Next, the above example demonstrates the clear method. The code is as follows:
public static void clearDemo()
{
Logger.debug("------------after clear------------------");
intBuffer.clear();
Logger.debug("position=" + intBuffer.position());
Logger.debug("limit=" + intBuffer.limit());
Logger.debug("capacity=" + intBuffer.capacity());
}
After the program runs, the results are as follows:
main |>empty clearDemo |> ------------after clear------------------ clearDemo |> position=0 clearDemo |> limit=20 clearDemo |> capacity=20
When the buffer is in read mode, call clear() and the buffer will be switched to write mode. After calling clear (), we can see that the value of position is cleared, that is, the starting position of writing is set to 0, and the upper limit of writing is the maximum capacity.
Basic steps of using Buffer class
Generally speaking, the basic steps of using Java NIO Buffer class are as follows:
(1) Create an instance object of the Buffer class using the allocate() method that creates the subclass instance object.
(2) Call the put method to write the data to the buffer.
(3) after writing, call Buffer. before reading data. Flip () method to convert the buffer to read mode.
(4) Call the get method to read data from the buffer.
(5) after reading, call Buffer.. Clear() or buffer The compact () method converts the buffer to write mode.
Explain the NIO Channel class in detail
As mentioned earlier, a connection in NIO is represented by a Channel. As we all know, from a broader perspective, a Channel can represent an underlying file descriptor, such as hardware device, file, network connection, etc. However, it is far more than that. In addition to corresponding to the underlying file descriptor, the Channel of Java NIO can be more detailed. For example, corresponding to different network transmission protocol types, there are different NIOChannel implementations in Java.
Main types of channels
Here, we will not describe too many complex Java NIO Channel types, but only focus on the four most important Channel implementations: FileChannel, SocketChannel, ServerSocketChannel and datagram Channel.
For the above four channels, the description is as follows:
(1) FileChannel is a file channel used for reading and writing data of files.
(2) SocketChannel is a Socket channel used for reading and writing data of Socket socket TCP connection.
(3) ServerSocketChannel server nested word channel (or server listening channel) allows us to listen to TCP connection requests and create a SocketChannel socket channel for each monitored request.
(4) Datagram channel is a datagram channel used for data reading and writing of UDP protocol.
These four channels cover file IO, TCP network, UDP IO and basic io.
The following is a brief introduction to the four channels from the four important operations of Channel acquisition, reading, writing and closing.
FileChannel
FileChannel is a channel dedicated to operating files.
Through FileChannel, you can read data from a file or write data to a file.
Specifically, FileChannel is in blocking mode and cannot be set to non blocking mode.
The following describes the four operations of FileChannel: get, read, write and close.
Get FileChannel channel
The FileChannel file channel can be obtained through the input stream and output stream of the file, for example:
//Create a file input stream FileInputStreamfis = new FileInputStream(srcFile); //Gets the channel of the file stream FileChannelinChannel = fis.getChannel(); //Create a file output stream FileOutputStreamfos = new FileOutputStream(destFile); //Gets the channel of the file stream FileChanneloutchannel = fos.getChannel();
You can also obtain FileChannel file channel through random access class of RandomAccessFile file:
// Create RandomAccessFile random access object
RandomAccessFileaFile = new RandomAccessFile("filename.txt","rw");
//Gets the channel of the file stream
FileChannelinChannel = aFile.getChannel();
Read FileChannel channel
In most application scenarios, when reading data from a channel, the int read (ByteBuffer buffer) method of the channel will be called. It reads the data from the channel, writes it to the ByteBuffer buffer, and returns the amount of data read.
RandomAccessFileaFile = new RandomAccessFile(fileName, "rw");
//Get channel
FileChannelinChannel=aFile.getChannel();
//Get a byte buffer
ByteBufferbuf = ByteBuffer.allocate(CAPACITY);
int length = -1;
//Call the read method of the channel to read the data and buy a buffer of byte type
while ((length = inChannel.read(buf)) != -1) {
//Omit Process the data in the read buf
}
Although it is read data for the channel, it is write data for the ByteBuffer buffer. At this time, the ByteBuffer buffer is in write mode.
Write to FileChannel channel
When writing data to the channel, in most application scenarios, the channel's intwrite (ByteBuffer buf fer) method will be called.
The parameter of this method, ByteBuffer buffer, is the source of data.
The write method reads data from the ByteBuffer buffer and writes it to the channel itself. The return value is the number of bytes successfully written.
//If the buf has just finished writing data, you need to flip the buf to turn it into the read mode
buf.flip();
int outlength = 0;
//Call the write method to write the data of buf to the channel
while ((outlength = outchannel.write(buf)) != 0) {
System.out.println("Bytes written:" + outlength);
}
Note that at this time, the ByteBuffer buffer is required to be readable and in read mode.
Close channel
When the channel is used, it must be closed. Closing is very simple. Just call the close method.
//Close channel channel.close();
Force refresh to disk
When the buffer is written to the channel, for performance reasons, the operating system cannot write data to disk in real time every time.
If you need to ensure that the buffered data written to the channel is actually written to the disk, you can call the force() method of FileChannel.
//Force refresh to disk channel.force(true);
Practice case of using FileChannel to complete file replication
The following is a simple practical case: copying files using file channels.
Its function is to use the FileChannel file channel to copy the original file, that is, to copy all the data in the original text to the target file. The complete code is as follows:
package com.crazymakercircle.iodemo.fileDemos;
import com.crazymakercircle.NioDemoConfig;
import com.crazymakercircle.util.IOUtil;
import com.crazymakercircle.util.Logger;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.nio.ByteBuffer;
import java.nio.channels.FileChannel;
/**
* Created by Neen @ crazy maker circle
*/
public class FileNIOCopyDemo
{
/**
* Entry function of demo program
*
* @param args
*/
public static void main(String[] args)
{
//Show me how to copy a resource file
nioCopyResouceFile();
}
/**
* Copy files under two resource directories
*/
public static void nioCopyResouceFile()
{
String sourcePath = NioDemoConfig.FILE_RESOURCE_SRC_PATH;
String srcPath = IOUtil.getResourcePath(sourcePath);
Logger.debug("srcPath=" + srcPath);
String destShortPath = NioDemoConfig.FILE_RESOURCE_DEST_PATH;
String destdePath = IOUtil.builderResourcePath(destShortPath);
Logger.debug("destdePath=" + destdePath);
nioCopyFile(srcPath, destdePath);
}
/**
* Copy file
*
* @param srcPath
* @param destPath
*/
public static void nioCopyFile(String srcPath, String destPath)
{
File srcFile = new File(srcPath);
File destFile = new File(destPath);
try
{
//If the target file does not exist, create a new one
if (!destFile.exists())
{
destFile.createNewFile();
}
long startTime = System.currentTimeMillis();
FileInputStream fis = null;
FileOutputStream fos = null;
FileChannel inChannel = null;
FileChannel outchannel = null;
try
{
fis = new FileInputStream(srcFile);
fos = new FileOutputStream(destFile);
inChannel = fis.getChannel();
outchannel = fos.getChannel();
int length = -1;
ByteBuffer buf = ByteBuffer.allocate(1024);
//Read buf from input channel
while ((length = inChannel.read(buf)) != -1)
{
//Flip buf to read mode
buf.flip();
int outlength = 0;
//Write buf to output channel
while ((outlength = outchannel.write(buf)) != 0)
{
System.out.println("Bytes written:" + outlength);
}
//Clear buf and change to write mode
buf.clear();
}
//Force refresh disk
outchannel.force(true);
} finally
{
IOUtil.closeQuietly(outchannel);
IOUtil.closeQuietly(fos);
IOUtil.closeQuietly(inChannel);
IOUtil.closeQuietly(fis);
}
long endTime = System.currentTimeMillis();
Logger.debug("base Replication milliseconds:" + (endTime - startTime));
} catch (IOException e)
{
e.printStackTrace();
}
}
}
In particular, in addition to the channel operation of FileChannel, you also need to pay attention to the mode switching of ByteBuffer.
The new ByteBuffer defaults to the write mode and can be used as an inchannel Parameter of read (ByteBuffer).
inChannel. The read method writes the data read from the channel inchannel to the ByteBuffer.
After that, you need to call the flip method of the buffer to switch the ByteBuffer to the read mode before it can be used as an outchannel The parameter of write (ByteBuffer) method, which reads data from ByteBuffer and then writes it to the output channel of outchannel. This completes a copy.
Before entering the next replication, the mode of the buffer should be switched again. After reading the ByteBuffer data, you need to switch to the write mode through the clear method before entering the next replication.
In the example code, each round of while loop in the outer layer requires two mode ByteBuffer switches: the first time, flip the buf to change to the read mode; During the second switching, clear buf and change to write mode.
The main purpose of the above example code is to demonstrate the use of file channels and byte buffers. As a file copying program, the efficiency of actual combat code is not the highest. For more efficient file replication, you can call the transferFrom method of the file channel. For the specific code, please refer to the fileniofastcopydemo class in the source code project. The path of the complete source file is: com crazymakercircle. iodemo. fileDemos. FileNIOFastCopyDemo
SocketChannel
In NIO, there are two channels involving network connection. One is SocketChannel, which is responsible for connection transmission, and the other is ServerSocketChannel, which is responsible for connection monitoring.
The SocketChannel transmission channel in NIO corresponds to the Socket class in OIO. The ServerSocketChannel listening channel in NIO corresponds to the ServerSocket class in OIO. ServerSocketChannel is used on the server side, while SocketChannel is on both the server side and the client side.
In other words, corresponding to a connection, there is a SocketChannel transmission channel responsible for transmission at both ends. Both ServerSocketChannel and SocketChannel support blocking and non blocking modes.
How to set the mode? Call the configureBlocking method as follows:
(1)socketChannel.configureBlocking (false) is set to non blocking mode.
(2)socketChannel.configureBlocking (true) is set to blocking mode. In the blocking mode, the connect, read and write operations of the SocketChannel channel are synchronous and blocking, and the efficiency is the same as the flow oriented blocking read and write operations of the old OIO in Java. Therefore, the specific operation of the channel in blocking mode is not introduced here. In the non blocking mode, the channel operation is asynchronous and efficient, which is also the advantage over the traditional OIO.
The following describes in detail the opening, reading, writing and closing operations of the channel in non blocking mode.
Get SocketChannel transport channel
On the client side, first obtain a socket transmission channel through the SocketChannel static method open(); Then, set the socket to non blocking mode; Finally, initiate a connection to the IP and port of the server through the connect() instance method.
//Get a socket transport channel
SocketChannelsocketChannel = SocketChannel.open();
//Set to non blocking mode
socketChannel.configureBlocking(false);
//Initiate a connection to the IP and port of the server
socketChannel.connect(new InetSocketAddress("127.0.0.1"，80));
In the case of blocking, the connection with the server may not be really established, socketchannel The connect method returns, so you need to spin constantly to check whether you are currently connected to the host:
while(! socketChannel.finishConnect() ){
//Keep spinning, waiting, or doing something else
}
On the server side, how to obtain the transmission socket? When a new connection event arrives, a new connection event can be successfully queried in the ServerSocketChannel on the server side, and the socket channel of the new connection can be obtained by calling the accept() method of the ServerSocketChannel listening socket on the server side:
//When a new connection event arrives, first obtain the server listening channel through the event ServerSocketChannel server = (ServerSocketChannel) key.channel(); //Gets the socket channel for the new connection SocketChannelsocketChannel = server.accept(); //Set to non blocking mode socketChannel.configureBlocking(false);
To emphasize, NIO socket channel is mainly used in non blocking application scenarios. Therefore, you need to call configure blocking (false) to set from blocking mode to non blocking mode.
Read SocketChannel transport channel
When the SocketChannel is readable, you can read data from the SocketChannel. The specific method is the same as the previous file channel reading method. Call the read method to read the data into the buffer ByteBuffer.
ByteBufferbuf = ByteBuffer.allocate(1024); int bytesRead = socketChannel.read(buf);
During reading, because it is asynchronous, we must check the return value of read to determine whether data has been read at present.
The return value of the read() method is the number of bytes read. If - 1 is returned, it means that the output end flag of the other party is read. The other party has finished the output and is ready to close the connection.
In fact, reading data through the read method itself is very simple. The more difficult thing is how to know when the channel is readable in the non blocking mode? This requires a new component of NIO - Selector channel Selector, which will be introduced later.
Write to SocketChannel transport channel
Like the previous file channel that writes data to FileChannel, most application scenarios will call the channel's int write (bytebufferbuffer) method.
//The buffer needs to be read before writing. ByteBuffer is required to be in read mode buffer.flip(); socketChannel.write(buffer);
Close the SocketChannel transport channel
Before closing the SocketChannel transmission channel, if the transmission channel is used to write data, it is recommended to call the shutdownOutput() method to terminate the output and send an output end flag (- 1) to the other party. Then call SocketChannel. base note. Close() method to close the socket connection.
//Terminate the output method and send an output end flag to the other party socketChannel.shutdownOutput(); //Close socket connection IOUtil.closeQuietly(socketChannel);
Practice case of sending files using SocketChannel
The following practice case is to use the FileChannel file channel to read the local file content, and then use the SocketChannel socket channel on the client to send the file information and file content to the server. The complete code of the client is as follows:
package com.crazymakercircle.iodemo.socketDemos;
import com.crazymakercircle.NioDemoConfig;
import com.crazymakercircle.util.IOUtil;
import com.crazymakercircle.util.Logger;
import java.io.File;
import java.io.FileInputStream;
import java.net.InetSocketAddress;
import java.nio.ByteBuffer;
import java.nio.channels.FileChannel;
import java.nio.channels.SocketChannel;
import java.nio.charset.Charset;
/**
* File transfer Client
* Created by Neen @ crazy maker circle
*/
public class NioSendClient
{
/**
* Constructor
* Establish a connection with the server
*
* @throws Exception
*/
public NioSendClient()
{
}
private Charset charset = Charset.forName("UTF-8");
/**
* Transfer files to the server
*
* @throws Exception
*/
public void sendFile()
{
try
{
String sourcePath = NioDemoConfig.SOCKET_SEND_FILE;
String srcPath = IOUtil.getResourcePath(sourcePath);
Logger.debug("srcPath=" + srcPath);
String destFile = NioDemoConfig.SOCKET_RECEIVE_FILE;
Logger.debug("destFile=" + destFile);
File file = new File(srcPath);
if (!file.exists())
{
Logger.debug("file does not exist");
return;
}
FileChannel fileChannel = new FileInputStream(file).getChannel();
SocketChannel socketChannel = SocketChannel.open();
socketChannel.socket().connect(
new InetSocketAddress(NioDemoConfig.SOCKET_SERVER_IP
, NioDemoConfig.SOCKET_SERVER_PORT));
socketChannel.configureBlocking(false);
Logger.debug("Client Successfully connected to the server");
while (!socketChannel.finishConnect())
{
//Keep spinning, waiting, or doing something else
}
//Send file name
ByteBuffer fileNameByteBuffer = charset.encode(destFile);
ByteBuffer buffer = ByteBuffer.allocate(NioDemoConfig.SEND_BUFFER_SIZE);
//Send file name length
int fileNameLen = fileNameByteBuffer.capacity();
buffer.putInt(fileNameLen);
buffer.flip();
socketChannel.write(buffer);
buffer.clear();
Logger.info("Client File name length sent:", fileNameLen);
//Send file name
socketChannel.write(fileNameByteBuffer);
Logger.info("Client File name sending completed:", destFile);
//Send file length
buffer.putLong(file.length());
buffer.flip();
socketChannel.write(buffer);
buffer.clear();
Logger.info("Client File length sending completed:", file.length());
//Send file content
Logger.debug("Start transferring files");
int length = 0;
long progress = 0;
while ((length = fileChannel.read(buffer)) > 0)
{
buffer.flip();
socketChannel.write(buffer);
buffer.clear();
progress += length;
Logger.debug("| " + (100 * progress / file.length()) + "% |");
}
if (length == -1)
{
IOUtil.closeQuietly(fileChannel);
socketChannel.shutdownOutput();
IOUtil.closeQuietly(socketChannel);
}
Logger.debug("======== File transfer succeeded ========");
} catch (Exception e)
{
e.printStackTrace();
}
}
/**
* entrance
*
* @param args
*/
public static void main(String[] args)
{
NioSendClient client = new NioSendClient(); // Start client connection
client.sendFile(); // transfer files
}
}
The file sending process in the above code: first send the target file name (without path), then send the file length, and finally send the file content. The configuration items in the code, such as server IP, server port, source file name to be sent (with path), remote target file name and other configuration information, are from system Read from the properties configuration file and complete the configuration through the customized NioDemoConfig configuration class.
Before running the above client program, you need to run the server program first. The server-side class and the client-side source code are in the same package. The class name is NioReceiveServer. Please refer to the source code project for details. We will introduce this class in detail later.
Datagram channel
Unlike the TCP transmission protocol of Socket socket, UDP protocol is not a connection oriented protocol. When using UDP protocol, as long as you know the IP and port of the server, you can send data directly to the other party. Using UDP protocol to transmit data in Java is simpler than TCP protocol. In Java NIO, datagram channel is used to handle the data transmission of UDP protocol.
Get datagram channel
The way to get datagram channel is very simple. Just call the open static method of datagram channel class. Then call configureBlocking (false) method and set it into non blocking mode. Base note:
//Get datagram channel DatagramChannel channel = DatagramChannel.open(); //Set to non blocking mode datagramChannel.configureBlocking(false);
If you need to receive data, you also need to call the bind method to bind the listening port of a datagram, as follows:
//Call the bind method to bind the listening port of a datagram channel.socket().bind(new InetSocketAddress(18080));
Read datagram channel data
When the datagram channel is readable, data can be read from the datagram channel. Different from the previous SocketChannel reading method, instead of calling the read method, call the receive (ByteBuffer buffer) method to read the data from the datagram channel and then write it to the ByteBuffer buffer.
//Create buffer ByteBufferbuf = ByteBuffer.allocate(1024); //Read from datagram channel and write to ByteBuffer buffer SocketAddressclientAddr= datagramChannel.receive(buffer);
The return value of the channel read receive (bytebufferbuffer) method is of type SocketAddress, which indicates that the connection address (including IP and port) of the sender is returned.
Reading data through the receive method is very simple, but how to know when the datagram channel is readable in non blocking mode? Like SocketChannel, NIO's new component Selector channel Selector is also needed, which will be introduced later.
Write datagram channel
Sending data to datagram channel is also different from sending data to SocketChannel. Instead of calling the write method, the send method is called. The example code is as follows:
//Flip buffer to read mode
buffer.flip();
//Call the send method to send the data to the target IP + port
dChannel.send(buffer,new InetSocketAddress(NioDemoConfig.SOCKET_SERVER_IP,
NioDemoConfig.SOCKET_SERVER_PORT));
//Clear the buffer and switch to write mode
buffer.clear();
Because UDP is a non connection oriented protocol, when calling the send method to send data, you need to specify the address (IP and port) of the receiver.
Close DatagramChannel datagram channel
This is relatively simple. Call the close() method directly to close the datagram channel.
//Simply close it dChannel.close()
Practice case of sending data using datagram channel packet channel
The following is a sample program code that uses datagram channel data packet to send data to the client. Its function is to obtain the user's input data and send the data to the remote server through datagram channel. The complete program code of the client is as follows:
package com.crazymakercircle.iodemo.udpDemos;
import com.crazymakercircle.NioDemoConfig;
import com.crazymakercircle.util.Dateutil;
import com.crazymakercircle.util.Print;
import java.io.IOException;
import java.net.InetSocketAddress;
import java.nio.ByteBuffer;
import java.nio.channels.DatagramChannel;
import java.util.Scanner;
/**
* create by Neen @ crazy maker circle
**/
public class UDPClient
{
public void send() throws IOException
{
//Operation 1: obtain datagram channel
DatagramChannel dChannel = DatagramChannel.open();
dChannel.configureBlocking(false);
ByteBuffer buffer = ByteBuffer.allocate(NioDemoConfig.SEND_BUFFER_SIZE);
Scanner scanner = new Scanner(System.in);
Print.tcfo("UDP Client started successfully!");
Print.tcfo("Please enter the content to send:");
while (scanner.hasNext())
{
String next = scanner.next();
buffer.put((Dateutil.getNow() + " >>" + next).getBytes());
buffer.flip();
// Operation 3: send data through datagram channel
dChannel.send(buffer,
new InetSocketAddress(NioDemoConfig.SOCKET_SERVER_IP
, NioDemoConfig.SOCKET_SERVER_PORT));
buffer.clear();
}
//Operation 4: close datagram channel
dChannel.close();
}
public static void main(String[] args) throws IOException
{
new UDPClient().send();
}
}
It can be seen from the sample program code that it is much simpler to send data through datagram channel on the client than using socket socket channel on the client.
Next, let's see how to use datagram channel packet channel to receive data on the server side?
The program code for the server to receive data through the datagram channel packet channel is posted below. You may not understand it at present, because the Selector selector is used in the code, but it doesn't matter. It will be introduced in the next section.
The receiving function of the server side is to bind a server address (IP + port) through the datagram channel datagram channel to receive UDP datagrams sent by the client. The complete code of the server side is as follows:
package com.crazymakercircle.iodemo.udpDemos;
import com.crazymakercircle.NioDemoConfig;
import com.crazymakercircle.util.Print;
import java.io.IOException;
import java.net.InetSocketAddress;
import java.net.SocketAddress;
import java.nio.ByteBuffer;
import java.nio.channels.DatagramChannel;
import java.nio.channels.SelectionKey;
import java.nio.channels.Selector;
import java.util.Iterator;
/**
* create by Neen @ crazy maker circle
**/
public class UDPServer
{
public void receive() throws IOException
{
//Operation 1: obtain datagram channel
DatagramChannel datagramChannel = DatagramChannel.open();
datagramChannel.configureBlocking(false);
datagramChannel.bind(new InetSocketAddress(
NioDemoConfig.SOCKET_SERVER_IP
, NioDemoConfig.SOCKET_SERVER_PORT));
Print.tcfo("UDP Server started successfully!");
Selector selector = Selector.open();
datagramChannel.register(selector, SelectionKey.OP_READ);
while (selector.select() > 0)
{
Iterator<SelectionKey> iterator = selector.selectedKeys().iterator();
ByteBuffer buffer = ByteBuffer.allocate(NioDemoConfig.SEND_BUFFER_SIZE);
while (iterator.hasNext())
{
SelectionKey selectionKey = iterator.next();
if (selectionKey.isReadable())
{
//Operation 2: read datagram channel data
SocketAddress client = datagramChannel.receive(buffer);
buffer.flip();
Print.tcfo(new String(buffer.array(), 0, buffer.limit()));
buffer.clear();
}
}
iterator.remove();
}
selector.close();
datagramChannel.close();
}
public static void main(String[] args) throws IOException
{
new UDPServer().receive();
}
}
Explain NIO Selector selector in detail
The three core components of Java NIO: channel, Buffer and Selector. Channel and Buffer are also closely related: data is always read from the channel to the Buffer, or written from the Buffer to the channel. So far, the first two components have been introduced. Now we welcome the last very important role - Selector.
Selector and registration
What is a Selector? What is the relationship between Selector and channel?
Simply put: the mission of the selector is to complete IO multiplexing. A channel represents a connection path, and the IO (input / output) status of multiple channels can be monitored at the same time through the selector.
The relationship between selector and channel is the relationship between monitoring and being monitored. The selector provides a unique API method to select which IO operation events are ready and ready for the monitored channel.
Generally speaking, a single thread handles a selector, and a selector can monitor many channels. Through selectors, a single thread can handle hundreds, thousands, tens of thousands, or even more channels. In extreme cases (tens of thousands of connections), only one thread can process all channels, which will greatly reduce the overhead of context switching between threads.
The relationship between channels and selectors is completed through register. Call the channel of the channel Register (Selector sel, int ops) method can register the channel instance into a selector.
The register method has two parameters: the first parameter specifies the selector instance to which the channel is registered; The second parameter specifies the type of IO event to be monitored by the selector. The channel IO event types that can be monitored by the selector include the following four types:
(1) Readable: selectionkey OP_ READ
(2) Writable: selectionkey OP_ WRITE
(3) Connection: selectionkey OP_ CONNECT
(4) Receive: SelectionKey OP_ The accept event type is defined in the SelectionKey class. If the selector wants to monitor multiple events of the channel, it can be implemented with the bitwise OR operator. For example, monitor both readable and writable IO events:
//Multiple events of the monitoring channel are realized by the bitwise OR operator int key = SelectionKey.OP_READ | SelectionKey.OP_WRITE ;
What is an IO event? This concept is easy to be confused. Here is a special explanation. The IO event here is not an IO operation on the channel, but a ready state of an IO operation of the channel, indicating that the channel has the conditions to complete an IO operation.
For example, when a SocketChannel completes the handshake connection with the opposite end, it is in the "OP_CONNECT" state. For another example, if a ServerSocketChannel listens to the arrival of a new connection, it is in the "OP_ACCEPT" state. For example, a SocketChannel channel with data readability is in the "OP_READ" state; A waiting to write data is in the "OP_WRITE" state.
selectableChannel to select a channel
Not all channels can be monitored or selected by the selector. For example, FileChannel file channels cannot be reused by selectors. To judge whether a channel can be monitored or selected by the selector, there is a premise: to judge whether it inherits the abstract class SelectableChannel (selectable channel). If it inherits SelectableChannel, it can be selected; otherwise, it cannot be selected. In short, if a channel can be selected, it must inherit the SelectableChannel class.
Where is the SelectableChannel class? It provides the public methods needed to realize the selectivity of channels. All network link Socket channels in Java NIO inherit the SelectableChannel class and are optional. The FileChannel file channel does not inherit the SelectableChannel, so it is not an optional channel.
SelectionKey selection key
After the monitoring relationship between the channel and the selector is registered successfully, you can select the ready event.
The specific selection work is completed by calling the select() method of the Selector. Through the select method, the Selector can continuously select the ready state of the operation in the channel and return the registered IO events of interest.
In other words, once some IO events occur in the channel (the ready state is reached) and are IO events registered in the selector, they will be selected by the selector and placed in the set of SelectionKey selection keys. A new concept, SelectionKey, appears here.
What is the SelectionKey? In short, the SelectionKey selection key is the IO event selected by the selector.
As mentioned earlier, after an IO event occurs (the ready state is reached), if it has been registered in the selector before, it will be selected by the selector and put into the SelectionKey selection key set; If you have not registered before, even if an IO event occurs, it will not be selected by the selector. The relationship between SelectionKey and IO can be simply understood as: selection key is the selected IO event. When programming, the function of selection key is very powerful. Through the SelectionKey selection key, you can not only obtain the IO event type of the channel, for example, SelectionKey OP_ READ； You can also obtain the channel where the IO event occurs; In addition, an example of a selector for selecting a selection key can also be obtained.
Selector usage process
There are three main steps to use the selector:
(1) Get the selector instance;
(2) Register the channel in the selector;
(3) Poll for IO ready events of interest (select key set).
Step 1: get the selector instance. The selector instance is obtained by calling the static factory method open(), as follows:
The selector using process mainly includes the following three steps:
(1) Get the selector instance;
(2) Register the selector into the channel;
(3) Poll for IO ready events of interest (select key set).
Step 1: get the selector instance
The selector instance is obtained by calling the static factory method open(), as follows:
//Call the static factory method open() to get the Selector instance .Selector selector = Selector.open();
The inner part of the class method open() of the Selector selector is to send a request to the Selector SPI (SelectorProvider) and obtain a new Selector instance through the default SelectorProvider (Selector provider) object.
The full name of SPI in Java is Service Provider Interface, which is an extensible service provision and discovery mechanism of JDK. Java provides the default implementation version of selector through SPI. In other words, other service providers can provide dynamic replacement or expansion of customized versions of selectors through SPI.
Step 2: register the channel with the selector
To implement selector management channel in the instance, you need to register the channel with the corresponding selector. The simple example code is as follows:
// 2. Access ServerSocketChannelserverSocketChannel = ServerSocketChannel.open(); // 3. Set to non blocking serverSocketChannel.configureBlocking(false); // 4. Binding connection serverSocketChannel.bind(new InetSocketAddress(SystemConfig.SOCKET_SERVER_PORT)); // 5. Register the channel on the selector and set the listening event as "receive connection" event serverSocketChannel.register(selector，SelectionKey.OP_ACCEPT);
The above registers the ServerSocketChannel channel to a selector by calling the register() method of the channel. Of course, before registering, you must first prepare the channel. Note here: the channel registered to the selector must be in non blocking mode, otherwise an IllegalBlockingModeException will be thrown.
This means that the FileChannel file channel cannot be used with the selector, because the FileChannel file channel has only blocking mode and cannot be switched to non blocking mode; All channels related to Socket can be.
Secondly, it should also be noted that one channel does not necessarily support all four IO events. For example, the server listens to the channel ServerSocketChannel, which only supports Accept IO events; The SocketChannel transport channel does not support Accept IO events.
How to determine which events are supported by the channel? Before registration, you can obtain all supported IO event sets of the channel through the validOps() method of the channel.
Step 3: select the IO ready event of interest (select key set)
Select the registered and ready IO events through the select() method of the Selector selector and save them to the SelectionKey selection key set.
SelectionKey collection is stored inside the selector instance. It is a collection (Set) whose element is of SelectionKey type. Call the selectedKeys() method of the selector to get the selection key collection.
Next, you need to iterate over each selection key of the set and perform the corresponding business operations according to the specific IO event type. The general processing flow is as follows:
//Poll to select the IO ready event of interest (select key set)
while (selector.select() > 0) {
Set selectedKeys = selector.selectedKeys();
Iterator keyIterator = selectedKeys.iterator();
while(keyIterator.hasNext()) {
SelectionKey key = keyIterator.next();
//Perform corresponding business operations according to specific IO event types
if(key.isAcceptable()) {
// IO event: ServerSocketChannel the server listens for a new connection to the channel
} else if (key.isConnectable()) {
// IO event: transmission channel connection succeeded
} else if (key.isReadable()) {
// IO event: transport channel readable
} else if (key.isWritable()) { Event transport channel writable
// IO event: transmission channel writable
}
//When processing is complete, remove the selection key
keyIterator.remove();
}
}
After processing, you need to remove the selection key from the SelectionKey set to prevent repeated processing in the next cycle. If you try to add a SelectionKey element to the collection, you cannot throw the SelectionKey element to the collection Lang.unsupported operationexception exception. The select() method used to select ready IO events has multiple overloaded implementation versions, as follows:
(1) select(): block the call until at least one channel has registered IO events.
(2) select(long timeout): the same as select(), but the maximum blocking time is the number of milliseconds specified by timeout.
(3) selectNow(): non blocking. It will be returned immediately regardless of whether there is an IO event or not. The integer value (int integer type) returned by the select() method indicates the number of channels where IO events have occurred. More precisely, it refers to how many channels have IO events from the last selection to this selection. Emphasize that the number returned by the select() method refers to the number of channels, not the number of IO events. Specifically, it refers to the number of channels in which the IO events of interest to the selector have occurred.
Practice case of using NIO to implement Discard server
The function of the Discard server is very simple: only read the input data of the client channel, and directly close the client channel after reading; And the read data is directly discarded. The Discard server is simple and clear enough. As the first communication example of learning NIO, it has more reference value. The following Discard server code refines the steps in the selector usage process:
package com.crazymakercircle.iodemo.NioDiscard;
import com.crazymakercircle.NioDemoConfig;
import com.crazymakercircle.util.Logger;
import java.io.IOException;
import java.net.InetSocketAddress;
import java.nio.ByteBuffer;
import java.nio.channels.SelectionKey;
import java.nio.channels.Selector;
import java.nio.channels.ServerSocketChannel;
import java.nio.channels.SocketChannel;
import java.util.Iterator;
public class NioDiscardServer
{
public static void startServer() throws IOException
{
// 1. Get Selector selector
Selector selector = Selector.open();
// 2. Get channel
ServerSocketChannel serverSocketChannel = ServerSocketChannel.open();
// 3. Set to non blocking
serverSocketChannel.configureBlocking(false);
// 4. Binding connection
serverSocketChannel.bind(new InetSocketAddress(NioDemoConfig.SOCKET_SERVER_PORT));
Logger.info("The server started successfully");
// 5. Register the channel on the selector and the registered IO event is: "receive new connection"
serverSocketChannel.register(selector, SelectionKey.OP_ACCEPT);
// 6. Poll for I/O ready events of interest (select key set)
while (selector.select() > 0)
{
// 7. Get selection key collection
Iterator<SelectionKey> selectedKeys = selector.selectedKeys().iterator();
while (selectedKeys.hasNext())
{
// 8. Get a single selection key and process it
SelectionKey selectedKey = selectedKeys.next();
// 9. Determine what event the key is
if (selectedKey.isAcceptable())
{
// 10. If the IO event of the selection key is a "connection ready" event, the client connection is obtained
SocketChannel socketChannel = serverSocketChannel.accept();
// 11. Switch to non blocking mode
socketChannel.configureBlocking(false);
// 12. Register the channel with the selector selector
socketChannel.register(selector, SelectionKey.OP_READ);
} else if (selectedKey.isReadable())
{
// 13. If the IO event of the selection key is a "readable" event, read the data
SocketChannel socketChannel = (SocketChannel) selectedKey.channel();
// 14. Read data
ByteBuffer byteBuffer = ByteBuffer.allocate(1024);
int length = 0;
while ((length = socketChannel.read(byteBuffer)) > 0)
{
byteBuffer.flip();
Logger.info(new String(byteBuffer.array(), 0, length));
byteBuffer.clear();
}
socketChannel.close();
}
// 15. Remove selection key
selectedKeys.remove();
}
}
// 7. Close connection
serverSocketChannel.close();
}
public static void main(String[] args) throws IOException
{
startServer();
}
}
The implementation of DiscardServer is divided into 16 steps, of which steps 7 to 15 are executed circularly. Continuously select the IO events of interest into the selection key set of the selector, and then use the selector Selectedkeys() gets the selection key set and performs iterative processing. For the newly established socket channel client transmission channel, it is also necessary to register on the same selector and use the same selection thread to continuously select the selection key for all registered channels.
In the DiscardServer program, two selector registrations are involved: one is to register the serverChannel server channel; Another time, register the received socket channel client transport channel. serverChannel is the newly connected IO event selectionkey registered by the server channel OP_ ACCEPT； The transmission channel of the client socketChannel is registered with the readable IO event selectionkey OP_ READ. When DiscardServer processes the selection key, it judges the type and then processes it accordingly
(1) If selectionkey OP_ Accept new connection event type, which represents that a new connection event has occurred in the serverChannel server channel, obtain the new socketChannel transmission channel through the accept method of the server channel, and register the new channel with the selector.
(2) If selectionkey OP_ Read readable event type, which means that if a client channel has data readable, the data of the socket channel transmission channel in the selection key is read and then discarded.
The DiscardClient code of the client is simpler. The client first establishes a connection to the server, sends some simple data, and then closes the connection directly. The code is as follows:
package com.crazymakercircle.iodemo.NioDiscard;
import com.crazymakercircle.NioDemoConfig;
import com.crazymakercircle.util.Logger;
import java.io.IOException;
import java.net.InetSocketAddress;
import java.nio.ByteBuffer;
import java.nio.channels.SocketChannel;
public class NioDiscardClient
{
/**
* client
*/
public static void startClient() throws IOException
{
InetSocketAddress address =
new InetSocketAddress(NioDemoConfig.SOCKET_SERVER_IP,
NioDemoConfig.SOCKET_SERVER_PORT);
// 1. Get channel
SocketChannel socketChannel = SocketChannel.open(address);
// 2. Switch to non blocking mode
socketChannel.configureBlocking(false);
//Keep spinning, waiting for the connection to complete, or do something else
while (!socketChannel.finishConnect())
{
}
Logger.info("Client connection succeeded");
// 3. Allocates a buffer of the specified size
ByteBuffer byteBuffer = ByteBuffer.allocate(1024);
byteBuffer.put("hello world".getBytes());
byteBuffer.flip();
socketChannel.write(byteBuffer);
socketChannel.shutdownOutput();
socketChannel.close();
}
public static void main(String[] args) throws IOException
{
startClient();
}
}
If you need to execute the whole program, first execute the previous server-side program, and then execute the subsequent client-side program. Through the development practice of Discard server, we should have a very clear understanding of the use process of NIO Selector.
Let's take a slightly more complicated case: receiving files and content on the server side.
A practical case of receiving files on the server side using SocketChannel
The receiving of this example demonstration file is a server-side program. And the SocketChannel client program for sending files described above are used together. Since the selector is needed on the server side, the Socket server-side program of NIO file transfer is introduced after introducing the selector. The example code of receiving files on the server side is as follows:
package com.crazymakercircle.iodemo.socketDemos;
import com.crazymakercircle.NioDemoConfig;
import com.crazymakercircle.util.IOUtil;
import com.crazymakercircle.util.Logger;
import com.crazymakercircle.util.Print;
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.net.InetSocketAddress;
import java.net.ServerSocket;
import java.nio.ByteBuffer;
import java.nio.channels.FileChannel;
import java.nio.channels.SelectableChannel;
import java.nio.channels.SelectionKey;
import java.nio.channels.Selector;
import java.nio.channels.ServerSocketChannel;
import java.nio.channels.SocketChannel;
import java.nio.charset.Charset;
import java.util.HashMap;
import java.util.Iterator;
import java.util.Map;
/**
* File transfer Server
* Created by Neen @ crazy maker circle
*/
public class NioReceiveServer
{
//Accept file path
private static final String RECEIVE_PATH = NioDemoConfig.SOCKET_RECEIVE_PATH;
private Charset charset = Charset.forName("UTF-8");
/**
* The client object saved on the server side corresponds to a client file
*/
static class Client
{
//File name
String fileName;
//length
long fileLength;
//Time to start transmission
long startTime;
//Address of the client
InetSocketAddress remoteAddress;
//Output file channel
FileChannel outChannel;
//Receiving length
long receiveLength;
public boolean isFinished()
{
return receiveLength >= fileLength;
}
}
private ByteBuffer buffer
= ByteBuffer.allocate(NioDemoConfig.SERVER_BUFFER_SIZE);
//Use Map to save each client transfer when op_ When the read channel is readable, find the corresponding object according to the channel
Map<SelectableChannel, Client> clientMap = new HashMap<SelectableChannel, Client>();
public void startServer() throws IOException
{
// 1. Get Selector selector
Selector selector = Selector.open();
// 2. Get channel
ServerSocketChannel serverChannel = ServerSocketChannel.open();
ServerSocket serverSocket = serverChannel.socket();
// 3. Set to non blocking
serverChannel.configureBlocking(false);
// 4. Binding connection
InetSocketAddress address
= new InetSocketAddress(NioDemoConfig.SOCKET_SERVER_PORT);
serverSocket.bind(address);
// 5. Register the channel on the selector and the registered IO event is: "receive new connection"
serverChannel.register(selector, SelectionKey.OP_ACCEPT);
Print.tcfo("serverChannel is linstening...");
// 6. Poll for I/O ready events of interest (select key set)
while (selector.select() > 0)
{
// 7. Get selection key collection
Iterator<SelectionKey> it = selector.selectedKeys().iterator();
while (it.hasNext())
{
// 8. Get a single selection key and process it
SelectionKey key = it.next();
// 9. Determine what event the key is and whether it is a new connection event
if (key.isAcceptable())
{
// 10. If the accepted event is a "new connection" event, get a new connection from the client
ServerSocketChannel server = (ServerSocketChannel) key.channel();
SocketChannel socketChannel = server.accept();
if (socketChannel == null) continue;
// 11. When the client is newly connected, switch to non blocking mode
socketChannel.configureBlocking(false);
// 12. Register the client's new connection channel with the selector
SelectionKey selectionKey =
socketChannel.register(selector, SelectionKey.OP_READ);
// The rest is business processing
Client client = new Client();
client.remoteAddress
= (InetSocketAddress) socketChannel.getRemoteAddress();
clientMap.put(socketChannel, client);
Logger.debug(socketChannel.getRemoteAddress() + "Connection successful...");
} else if (key.isReadable())
{
processData(key);
}
// NIO features will only be accumulated, and the set of selected keys will not be deleted
// If it is not deleted, it will be selected by the select function next time
it.remove();
}
}
}
/**
* Process the data transmitted from the client
*/
private void processData(SelectionKey key) throws IOException
{
Client client = clientMap.get(key.channel());
SocketChannel socketChannel = (SocketChannel) key.channel();
int num = 0;
try
{
buffer.clear();
while ((num = socketChannel.read(buffer)) > 0)
{
buffer.flip();
//The file name is processed first when it is sent from the client
if (null == client.fileName)
{
if (buffer.capacity() < 4)
{
continue;
}
int fileNameLen = buffer.getInt();
byte[] fileNameBytes = new byte[fileNameLen];
buffer.get(fileNameBytes);
// file name
String fileName = new String(fileNameBytes, charset);
File directory = new File(RECEIVE_PATH);
if (!directory.exists())
{
directory.mkdir();
}
Logger.info("NIO Transmission target dir: ", directory);
client.fileName = fileName;
String fullName = directory.getAbsolutePath() + File.separatorChar + fileName;
Logger.info("NIO Transfer destination file:", fullName);
File file = new File(fullName.trim());
if (!file.exists())
{
file.createNewFile();
}
FileChannel fileChannel = new FileOutputStream(file).getChannel();
client.outChannel = fileChannel;
if (buffer.capacity() < 8)
{
continue;
}
// file length
long fileLength = buffer.getLong();
client.fileLength = fileLength;
client.startTime = System.currentTimeMillis();
Logger.debug("NIO Transmission start:");
client.receiveLength += buffer.capacity();
if (buffer.capacity() > 0)
{
// write file
client.outChannel.write(buffer);
}
if (client.isFinished())
{
finished(key, client);
}
buffer.clear();
}
//The client sends it, and finally the file content
else
{
client.receiveLength += buffer.capacity();
// write file
client.outChannel.write(buffer);
if (client.isFinished())
{
finished(key, client);
}
buffer.clear();
}
}
key.cancel();
} catch (IOException e)
{
key.cancel();
e.printStackTrace();
return;
}
// Call close to - 1 to reach the end
if (num == -1)
{
finished(key, client);
buffer.clear();
}
}
private void finished(SelectionKey key, Client client)
{
IOUtil.closeQuietly(client.outChannel);
Logger.info("Upload complete");
key.cancel();
Logger.debug("File received successfully,File Name: " + client.fileName);
Logger.debug(" Size: " + IOUtil.getFormatFileSize(client.fileLength));
long endTime = System.currentTimeMillis();
Logger.debug("NIO IO Transmission milliseconds:" + (endTime - client.startTime));
}
/**
* entrance
*
* @param args
*/
public static void main(String[] args) throws Exception
{
NioReceiveServer server = new NioReceiveServer();
server.startServer();
}
}
Because each time the client transfers files, it will be divided into multiple transfers:
(1) First pass in the file name.
(2) The second is the file size.
(3) Then there is the content of the file. Corresponding to each Client socketChannel, create a Client object to save the Client state, save the file name, file size and written target file channel outChannel respectively.
There is a one-to-one correspondence between socketChannel and Client object: when establishing a connection, take socketChannel as the Key and Client object as the Value, and save the Client in the map. When the socketChannel transmission channel has data readable, select Key Channel() method to retrieve the socketChannel channel where the IO event is located. Then get the corresponding Client object from the map through the socketChannel channel. When receiving data, if the file name is empty, first process the file name, save the file name to the Client object, and create the target file on the server; Next, read the data again, indicating that the file size has been received, and save the file size to the Client object; Next, if the data is received, it indicates that it is the file content, then it is written to the outChannel file channel of the Client object until the data is read.
Operation mode: after starting the NioReceiveServer server program, start the client program NioSendClient described above, that is, the file transfer can be completed.
Summary chapter
In terms of programming difficulty, Java NIO programming is much more difficult than synchronous blocking Java OIO programming. Please note that the previous practice case is relatively simple, not a complex communication program, and there are no problems such as "sticking package" and "unpacking". If these problems are added, the code will be more complex.
Compared with Java OIO, the general characteristics of Java NIO programming are as follows:
(1) In NIO, the server receives new connections asynchronously. Unlike Java's OIO, the server listens for connections, which are synchronous and blocked. NIO can continuously poll the selection key set of the selector through the selector (also known as multiplexer) to select the new connection.
(2) In NIO, the read and write operations of the SocketChannel transmission channel are asynchronous. If there is no read-write data, the thread responsible for IO communication will not wait synchronously. In this way, the thread can handle other connected channels; There is no need to block the thread like OIO until the responsible connection is available.
(3) In NIO, a selector thread can handle thousands of client connections at the same time, and the performance will not decline linearly with the increase of clients.
In short, with the epoll support at the bottom of Linux and the application layer IO reuse technology such as Java NIO Selector selector, Java programs can realize high TPS and high concurrency of IO communication, and make the server have the connection ability of hundreds of thousands and millions of concurrency.
Java NIO technology is very suitable for high-performance and high load network servers. The famous communication server middleware Netty is implemented based on NIO technology of Java. Of course, Java NIO technology is only the foundation. If you want to achieve high performance and high concurrency of communication, you can't do without efficient design patterns.