I checked a lot of groupcache-related information on the Internet, but most of them were copy, and there were no actual examples, so I looked at the source code, asked on golang-nuts, and searched for some use of groupcache on github For example, here is a summary. At present, this cache library is only in the state of understanding
general introduction
In fact, there are a lot of introductions about groupcache on the Internet, and all of them are introductions. Of course, there are also pictures how to deploy them, but the text and pictures are not in the same time and space at all, and the pictures are also copied from foreign countries. blog . It does not have a server like some other cache databases, which requires a client to connect. The document clearly states that it is a program library, so there is no such thing as a server and a client. In other words, it does not have a server. , or everyone is a server, the feeling of God of Cookery arises spontaneously.
main code structure
Its code structure is also relatively clear, and the amount of code is not very large, which is very suitable for everyone to read and study. It is mainly divided into consistenthash (provides support for consistent hash algorithm), lru (provides an algorithm for clear cache in LRU mode), singleflight (guarantees that multiple identical requests only get the value once, reducing resource consumption), and some Source file: byteview.go provides a container similar to a data, http.go provides the implementation of cache communication between different addresses, peers.go node definition, sinks.go feels like opening up space for the container and interacting with the container A middleman of groupcache.go, the leader in the entire source code of groupcache.go, and others serve it.
a test example
The more important aspects when using this cache are also the places where I made mistakes before.
- Two places need to be monitored, one is to listen to nodes, and the other is to listen to requests
- When setting node addresses in batches, you need to add http:// in front of the address, because I didn’t add it at the beginning, so the cache information can’t be interacted with nodes.
- The started node address should be consistent with the set node address: quantity and address value. Because I can't interact between nodes every time I lose one.
Some of the above information may also be wrong, it is just the result of my personal test.
code part
package main import ( "flag" "fmt" "github.com/golang/groupcache" "io/ioutil" "log" "net/http" "os" "strconv" "strings" ) var ( // peers_addrs = []string{"127.0.0.1:8001", "127.0.0.1:8002", "127.0.0.1:8003"} //rpc_addrs = []string{"127.0.0.1:9001", "127.0.0.1:9002", "127.0.0.1:9003"} index = flag.Int("index", 0, "peer index") ) func main() { flag.Parse() peers_addrs := make([]string, 3) rpc_addrs := make([]string, 3) if len(os.Args) > 0 { for i := 1; i < 4; i++ { peers_addrs[i-1] = os.Args[i] rpcaddr := strings.Split(os.Args[i], ":")[1] port, _ := strconv.Atoi(rpcaddr) rpc_addrs[i-1] = ":" + strconv.Itoa(port+1000) } } if *index < 0 || *index >= len(peers_addrs) { fmt.Printf("peer_index %d not invalid\n", *index) os.Exit(1) } peers := groupcache.NewHTTPPool(addrToURL(peers_addrs[*index])) var stringcache = groupcache.NewGroup("SlowDBCache", 64<<20, groupcache.GetterFunc( func(ctx groupcache.Context, key string, dest groupcache.Sink) error { result, err := ioutil.ReadFile(key) if err != nil { log.Fatal(err) return err } fmt.Printf("asking for %s from dbserver\n", key) dest.SetBytes([]byte(result)) return nil })) peers.Set(addrsToURLs(peers_addrs)...) http.HandleFunc("/zk", func(rw http.ResponseWriter, r *http.Request) { log.Println(r.URL.Query().Get("key")) var data []byte k := r.URL.Query().Get("key") fmt.Printf("cli asked for %s from groupcache\n", k) stringcache.Get(nil, k, groupcache.AllocatingByteSliceSink(&data)) rw.Write([]byte(data)) }) go http.ListenAndServe(rpc_addrs[*index], nil) rpcaddr := strings.Split(os.Args[1], ":")[1] log.Fatal(http.ListenAndServe(":"+rpcaddr, peers)) } func addrToURL(addr string) string { return "http://" + addr } func addrsToURLs(addrs []string) []string { result := make([]string, 0) for _, addr := range addrs { result = append(result, addrToURL(addr)) } return result }
Execution method: ./demo 127.0.0.1:8001 127.0.0.1:8002 127.0.0.1:8003
In the above command, we started a node, and set the number of node addresses to 3. Here, since my default index is 0, when starting other nodes, change the order of the addresses so that the first address It's okay to be different three times. (Sorry, the implementation here is not very good), so start three nodes in the same way.
Open the browser and access 127.0.0.1:9001?key=1.txt, where 1.txt is the place where data needs to be obtained, similar to the actual database, I directly replace it with a file here.
operation result:
- When I access :9001 the result
In the above figure, we are like: 8001 This node requests data, it has no cached data, and then it will go to other nodes to find the data of this key, and when there is no data, it will go to the database to get it (here we are a file), Therefore, the operation of obtaining database data will appear in the node: 8003.
- Then I visit: 9002
According to the figure above, the first address is: 8002. This node directly gets the value from the cache, but this node does not cache data before the request, which is also completed by the interaction between nodes.
I opened two virtual machines locally, and in the same local area network, the test can also get the same result. The results here will not be pasted anymore. Of course, the node address should be changed accordingly when running, according to the address in the LAN of each machine.
This blog has relatively few introductions and source code descriptions about groupcache. The main thing is to post a test example, because I saw very few on the Internet, and it did not give how to run or run results at all (not including just mentioned I found a blog by a foreigner, he wrote it very well, I just read his blog before starting to write it myself).