Chen Hengjie is the editor in chief of TesterHome community and the producer of the 10th MTSC Conference Shanghai Station - open source. He has successively engaged in the work related to the improvement of test efficiency in PP assistant, PPmoney, litchi and other companies, and has rich experience in the improvement of test technology and efficiency.
stay A taste of JVM sandbox repeater, a general traffic recording and playback tool (II) -- use of repeater console You can see that the core of the repeater is still in the plugin, so it is necessary to learn.
Familiar with JVM sandbox
The underlying layer of repeater plugin involves some principles of JVM sandbox. You need to read the following related documents first:
- Overall introduction: https://github.com/alibaba/jv...
- Detailed wiki documents: https://github.com/alibaba/jv...
repeater itself is a module of JVM sandbox, so it focuses on two chapters: users and module developers. Since this part is not the focus of this paper, only the parts that are more relevant to the summary record and this paper.
- JVM sandbox is a JVM sandbox container, a non intrusive runtime AOP solution for JVM. By abstracting BEFORE, RETURN, thread and other events, the specified class can be enhanced and modified at runtime
- JVM sandbox supports two modes: attach and Java agent. Our previous repeater example uses attach
- The repeater belongs to the user of the JVM sandbox_ Module, so put it in ${home} / Sandbox module /
- sandbox.sh is the main operation client of sandbox. In addition to the attach command we used, it also includes refresh user module (- F), force refresh user module (- F), reset (- R), and close container (- S). To stop the attach in the future, you can directly use - S
- The sandbox itself contains the http module, so our console can communicate with the repeater module in the sandbox through http (that is, the interface for transmitting _data field mentioned in the playback method in the repeater user document)
It is strongly recommended that you do it yourself Module writing elementary , about 15 minutes. Don't copy and paste the code, but type it out by imitating the document, so the memory is more profound.
At the same time, you can refer to How to debug? This issue, learn how to debug the JVM sandbox module. The subsequent debugging of repeater plugin can also be used.
Introduction to repeater plugin
In particular, considering that the goal of the survey is to use, we should use a certain degree before considering a deeper understanding. Therefore, for the time being, skip the resolution of repeater module and its related dependencies. Subsequent replenishment.
There is no official introduction in the official documents, so please sort it out according to your personal understanding.
- The repeater itself mainly provides a mechanism to capture the input and output parameters of various methods in the jvm. However, there are many internal methods in an application, so it is impossible and unnecessary to capture the access and exit of all methods. Therefore, it is necessary to filter and provide different implementations according to different methods (such as whether it supports playback and mock)
- In order to increase these supports, plugin is a better way, which is simple to implement and easy to expand.
- Each plugin needs to complete three things: it can identify itself according to the specification, realize the record of specified input and output parameters, and realize playback (optional)
At present, the list of plug-ins officially provided is as follows (as of 20190717):
| Plug in type | Recording | playback | Mock | Support time | contributor |
|---|---|---|---|---|---|
| http-plugin | √ | √ | × | 201906 | zhaoyb1990 |
| dubbo-plugin | √ | × | √ | 201906 | zhaoyb1990 |
| ibatis-plugin | √ | × | √ | 201906 | zhaoyb1990 |
| mybatis-plugin | √ | × | √ | 201906 | ztbsuper |
| java-plugin | √ | √ | √ | 201906 | zhaoyb1990 |
| redis-plugin | × | × | × | Expected by the end of July | NA/NA |
Read the plugin source code
Similarly, there are three steps to read the source code: clarify the reading purpose, understand the overall architecture, and carefully read the target functions
step 0 clarify the purpose of reading
Learn the steps of plugin development and complete the design and development of a rabbitmq plugin.
Understand the overall architecture of step 1
Let's take a look at the structure of each plugin and see if there are some common features:
$ tree -L 12 repeater-plugins | grep -v iml | grep -v target repeater-plugins ├── dubbo-plugin │ ├── pom.xml │ └── src │ └── main │ └── java │ └── com │ └── alibaba │ └── jvm │ └── sandbox │ └── repeater │ └── plugin │ └── dubbo │ ├── DubboConsumerPlugin.java │ ├── DubboProcessor.java │ ├── DubboProviderPlugin.java │ └── DubboRepeater.java ├── http-plugin │ ├── pom.xml │ └── src │ └── main │ └── java │ └── com │ └── alibaba │ └── jvm │ └── sandbox │ └── repater │ └── plugin │ └── http │ ├── HttpPlugin.java │ ├── HttpRepeater.java │ ├── HttpStandaloneListener.java │ ├── InvokeAdvice.java │ └── wrapper ├── ibatis-plugin │ ├── pom.xml │ └── src │ └── main │ └── java │ └── com │ └── alibaba │ └── jvm │ └── sandbox │ └── repeater │ └── plugin │ └── ibatis │ ├── IBatisPlugin.java │ └── IBatisProcessor.java ├── java-plugin │ ├── pom.xml │ └── src │ └── main │ └── java │ └── com │ └── alibaba │ └── jvm │ └── sandbox │ └── repeater │ └── plugin │ └── java │ ├── JavaEntrancePlugin.java │ ├── JavaInvocationProcessor.java │ ├── JavaPluginUtils.java │ ├── JavaRepeater.java │ └── JavaSubInvokePlugin.java ├── mybatis-plugin │ ├── pom.xml │ └── src │ └── main │ └── java │ └── com │ └── alibaba │ └── jvm │ └── sandbox │ └── repeater │ └── plugin │ └── mybatis │ ├── MybatisPlugin.java │ └── MybatisProcessor.java ├── pom.xml ├── redis-plugin │ ├── pom.xml │ └── src │ └── main │ └── java │ └── com │ └── alibaba │ └── jvm │ └── sandbox │ └── repeater │ └── plugin │ └── redis │ ├── RedisPlugin.java │ └── RedisProcessor.java
As can be seen from the above, the basic structure has two categories.
- One is a simple plug-in represented by mybatis plugin. You only need to implement a plugin class and a processor class. The official manual examples are also of this kind.
- One is the complex plug-ins represented by Java plugin and HTTP plugin. In addition to the plugin class, there are other auxiliary classes.
For ease of understanding, start with a simple. First look at mybatis plugin.
├── mybatis-plugin │ ├── pom.xml │ └── src │ └── main │ └── java │ └── com │ └── alibaba │ └── jvm │ └── sandbox │ └── repeater │ └── plugin │ └── mybatis │ ├── MybatisPlugin.java // The class implementing InvokePlugin SPI mainly identifies the java class to be monitored and some basic information (name, data type, etc.) of the plug-in │ └── MybatisProcessor.java // The class that implements InvocationProcessor interface processing calls mainly provides the assembly and implementation of Identity and request.
Well, let's look at the more complex HTTP plugin
$ tree -L 12 | grep -v iml | grep -v target
.
├── pom.xml
└── src
└── main
└── java
└── com
└── alibaba
└── jvm
└── sandbox
└── repater
└── plugin
└── http
├── HttpPlugin.java // The class implementing InvokePlugin SPI can be understood as an overall entry, similar to the Application of Spring
├── HttpRepeater.java // Implement Repeater and support the core class of playback
├── HttpStandaloneListener.java // For the special implementation of standalone mode, it mainly supports header transparent transmission of traceId
├── InvokeAdvice.java // http request aware interface, including synchronous and asynchronous calls
└── wrapper
├── WrapperAsyncListener.java // An implementation of AsyncListener, which is mainly used to deal with asynchronous requests?
├── WrapperOutputStreamCopier.java // A class copied by the output stream has no logic. It feels like a tool class
├── WrapperRequest.java // An implementation class of HttpServletRequestWrapper, which changes request into a custom servlet to facilitate custom implementation
├── WrapperResponseCopier.java // The implementation class of HttpServletResponseWrapper changes the response into a custom servlet to facilitate custom implementation
└── WrapperTransModel.java // An entity class that contains request, response, url, etc., and provides a constructor with an input parameter of WrapperRequest object. The effect is unknown.To summarize:
- Each plugin must have a class of xxPlugin to implement InvokePlugin SPI.
- Most plugins need to have an xxProcessor class to implement InvocationProcessor. At present, there is only HTTP plugin exception.
- A small number of plug-ins supporting playback (entry call class plug-ins) need to have a class of xxRepeater to provide the corresponding implementation.
step 2 detailed reading target function
As you can see in the previous step, plugin and processor are relatively more common implementations. So focus on this.
Here, mybatis plugin is used as the representative for parsing.
- MybatisPlugin
@MetaInfServices(InvokePlugin.class) // Indicates that it is a plug-in SPI
public class MybatisPlugin extends AbstractInvokePluginAdapter {
@Override
protected List<EnhanceModel> getEnhanceModels() { // Define an EnhanceModel to mark which classes and events need to be monitored
EnhanceModel em = EnhanceModel.builder()
.classPattern("org.apache.ibatis.binding.MapperMethod") // The class name to be monitored is org apache. ibatis. binding. MapperMethod
.methodPatterns(EnhanceModel.MethodPattern.transform("execute")) // The method to be monitored is called execute
.watchTypes(Type.BEFORE, Type.RETURN, Type.THROWS) // Listening events. Listen here for BEFORE (just BEFORE entering the method and calling the actual logic), RETURN (when the calling logic ends, the RETURN value is ready and ready to RETURN upward), and THROWS (when an exception is found, the exception is ready and ready to throw upward)
.build();
return Lists.newArrayList(em);
}
@Override
protected InvocationProcessor getInvocationProcessor() { // Implement the method that returns InvocationProcessor.
return new MybatisProcessor(getType()); // The plug-in itself has its own Processor, so the plug-in's own Processor is returned
}
@Override
public InvokeType getType() { // Set InvokeType to MYBATIS. This is used to identify what type of call is recorded. The repeater will select the corresponding plug-in for playback or mock according to the type of recorded message
return InvokeType.MYBATIS;
}
@Override
public String identity() { // Set a unique identification name. When you start loading a plug-in, there will be a log print name of the loaded plug-in, which comes from here. Therefore, we need to be unique.
return "mybatis";
}
@Override
public boolean isEntrance() { // Whether to check the inlet flow plug-in.
return false;
}
}- MybatisProcessor
class MybatisProcessor extends DefaultInvocationProcessor {
MybatisProcessor(InvokeType type) {
super(type);
}
/**
* Assembly identification
* @param event The BeforeEvent object obtained from sandbox records the relevant information of this event
* @return An Identity object is used as the identification of traffic
*/
@Override
public Identity assembleIdentity(BeforeEvent event) {
// Gets the object that triggers the call event. In short, the object to which the method belongs is currently intercepted
Object mapperMethod = event.target;
// SqlCommand = MapperMethod.command
// Get the command field in the class corresponding to this object
Field field = FieldUtils.getDeclaredField(mapperMethod.getClass(), "command", true);
// If the obtained value is null, set location (the second parameter) and endpoint (the third parameter) to "Unknown", assemble Identity and return.
if (field == null) {
return new Identity(InvokeType.MYBATIS.name(), "Unknown", "Unknown", new HashMap<String, String>(1));
}
try {
// Get the object corresponding to the "command" field in the trigger call event object and coexist it in the variable command
Object command = field.get(mapperMethod);
// Call getName and getType methods of variable command respectively
Object name = MethodUtils.invokeMethod(command, "getName");
Object type = MethodUtils.invokeMethod(command, "getType");
// Use type Tostring() as location, name Tostring() is used as the endpoint to assemble Identity and return
return new Identity(InvokeType.MYBATIS.name(), type.toString(), name.toString(), new HashMap<String, String>(1));
} catch (Exception e) {
// In case of any exception, set location (the second parameter) and endpoint (the third parameter) to Unknown, assemble Identity and return.
return new Identity(InvokeType.MYBATIS.name(), "Unknown", "Unknown", new HashMap<String, String>(1));
}
}
@Override
public Object[] assembleRequest(BeforeEvent event) {
// MapperMethod#execute(SqlSession sqlSession, Object[] args)
// args may have a non serializable exception (e.g. using tk.mybatis)
// The implementation provided by the default parent class is to return the whole event Argumentarray, the implementation here changes it to return only the elements with subscript 1 and remove other elements. From the annotation point of view, we want to avoid the non serializable exception of subsequent args, but from the implementation point of view, we take the second element instead of the first parameter. The reason is unknown.
return new Object[]{event.argumentArray[1]};
}
}A brief summary is as follows:
- plugin mainly listens to org apache. ibatis. binding. The execute method of mappermethod class will listen to BEFORE, AFTER and thread events.
- During execution, it will try to add the type and name attributes to the traffic ID by obtaining the command attribute value of the object of MapperMethod class. Otherwise, fill in with unknown.
Then why do you do the above operation? Let's take a look at the code snippet of the execute method of MapperMethod class in mybatis.
public class MapperMethod {
// This is the command object we want to get after labeling
private final SqlCommand command;
private final MethodSignature method;
public MapperMethod(Class<?> mapperInterface, Method method, Configuration config) {
this.command = new SqlCommand(config, mapperInterface, method);
this.method = new MethodSignature(config, mapperInterface, method);
}
// This is the execute method we want to capture
public Object execute(SqlSession sqlSession, Object[] args) {
Object result;
// The type of command represents the database operation type, corresponding to five types: add, delete, modify, query, and flush
switch (command.getType()) {
case INSERT: {
Object param = method.convertArgsToSqlCommandParam(args);
result = rowCountResult(sqlSession.insert(command.getName(), param));
break;
}
case UPDATE: {
Object param = method.convertArgsToSqlCommandParam(args);
result = rowCountResult(sqlSession.update(command.getName(), param));
break;
}
case DELETE: {
Object param = method.convertArgsToSqlCommandParam(args);
result = rowCountResult(sqlSession.delete(command.getName(), param));
break;
}
case SELECT:
if (method.returnsVoid() && method.hasResultHandler()) {
executeWithResultHandler(sqlSession, args);
result = null;
} else if (method.returnsMany()) {
result = executeForMany(sqlSession, args);
} else if (method.returnsMap()) {
result = executeForMap(sqlSession, args);
} else if (method.returnsCursor()) {
result = executeForCursor(sqlSession, args);
} else {
Object param = method.convertArgsToSqlCommandParam(args);
result = sqlSession.selectOne(command.getName(), param);
}
break;
case FLUSH:
result = sqlSession.flushStatements();
break;
default:
throw new BindingException("Unknown execution method for: " + command.getName());
}
if (result == null && method.getReturnType().isPrimitive() && !method.returnsVoid()) {
throw new BindingException("Mapper method '" + command.getName()
+ " attempted to return null from a method with a primitive return type (" + method.getReturnType() + ").");
}
return result;
}
...It can be seen from the code that this execute serves as a connecting link between the preceding and the following. Basically, all database operations will pass through here, and there is enough information to make the unique identification of a single call.
As the official document says, as long as you find the right class to capture, the rest will go smoothly.
But there is one unsolved mystery:
1. Why should assemblyrequest adjust the return to return only the second input parameter?
Official reply:
The first parameter of execute is SqlSession, which does not need or cannot be serialized, and has no meaning for recording and playback. Assemblyrequest itself is also used as request processing, and some parameters do not necessarily need to be used.
Start developing rabbitmq plug-ins
After the above interpretation, the development scheme is relatively clear.
1. rabbitmq can be called in two main ways. One is the producer, which generates mq information as a sub call and sends it to the queue. The other is the consumer, which triggers the subsequent logic as an entry call. In recording and playback, consumers' scenes are more important and need to be met first. In this scenario, playback is required.
2. You need to find rabbitmq as the connecting class and method of consumers, and correspond to RabbitMqPlugin.
3. Implement the corresponding processor class and repeater class to realize playback.
For more details, it will be supplemented after subsequent completion.
This article was first published in the TesterHome community, Click this link to view the original text and communicate directly with the author.
Today's knowledge has been absorbed~
To learn more about cutting-edge test development technologies: welcome to "the 10th MTSC Conference Shanghai station" > > >
1 main venue + 12 special sessions, where celebrities gather and elites gather