appmesh configuration ecs

References

• https://docs.amazonaws.cn/app-mesh/latest/userguide/getting-started-ecs.html

Action required:

• Authorize the envoy agent deployed in the ecs task to read the virtual node configuration
• Update ecs task definition to use envoy proxy
• Use xray to track envoy requests

be careful:

• The envoy container needs credentials to communicate with the appmesh service (ec2 type can use metadata or task roles, fargate can only use task roles)

• Create a task definition to integrate appmesh, enable appmesh, associate with the previously created mesh, and make sure to use the awsvpc network mode

• In the task definition, the container must wait for the envoy agent to boot and start before it can start. Set the dependon parameter

appmesh examples repository, https://github.com/aws/aws-app-mesh-examples

How to use the China repository:

• Modify the resource to aws-cn
• Modify the terminal node to amazonaws.com.cn
• Bypass 80/8080/443 ports

The architecture of this deployment is similar to the previous ec2 deployment appmesh, Refer to this article for details

create ecs application

Configure ecs task definition

We use fargate to deploy the test service. For convenience, we use the amazonlinux image (preferably packaged into an image after manually installing the tool), and enable the ecs exec function

yum install procps-ng -y
yum install bind-utils -y
yum install nginx -y
yum install telnet -y
docker commit c6d40d5fb469 xxxxxx.dkr.ecr.cn-north-1.amazonaws.com.cn/amazonlinux
docker push xxxxxx.dkr.ecr.cn-north-1.amazonaws.com.cn/amazonlinux:latest

https://docs.amazonaws.cn/en_us/AmazonECS/latest/developerguide/ecs-exec.html

Note: ECS Exec cannot be turned on for existing tasks, it can only be turned on for new tasks, and the session idle timeout is 20 minutes

Certain permissions need to be configured for the ecs task role

In order to use appmesh in ecs, the following conditions need to be met

• Make sure that the appmesh container has been configured in the ecs task definition
• Make sure to use service discovery. In fact, you can find the service through dns. You can use coredns directly in eks. You need to manually configure dns resolution (cloudmap) on ec2. ecs can automatically register cloudmap by enabling service discovery

The process of ecs service configuration service discovery

When creating a service, when it comes to load balancer, service discovery, automatic scaling, creation and deletion of network cards, cw alarm log and ecs exec, you need to use the ecs service role, and the trusted object is ecs.amazonaws.com service

https://docs.aws.amazon.com/cli/latest/reference/ecs/create-service.html

The control service discovery is the --service-registries parameter, with the following precautions

https://docs.amazonaws.cn/AmazonECS/latest/developerguide/service-discovery.html

• The number of tasks for each service in service discovery is limited to 1000
• Only supports registering services with private dns namespaces
• Need to configure dns resolution of vpc
• Automatically created cloudmap resources need to be manually deleted

Note: The following cli operations are just to understand the entire process of service discovery and creation. In fact, if you use the console, the ns and service of the cloudmap will be automatically created without manual configuration.

Use the command line to create a cloudmap. Note that the command used here is servicediscovery, create a namespace named ecs-mesh, and create a private managed domain of route53 by default.

aws servicediscovery create-private-dns-namespace \
    --name ecs-mesh.local \
    --vpc vpc-086d798b56f59e2ae

If the ecs service created later specifies a namespace, it will automatically register the service name to the hosting domain

Created private hosting domain, note that this hosting domain cannot be edited directly, it can only be set through cloudmap

Create servicea, this namespace-id is the namespaceid of cloudmap

aws servicediscovery create-service \
    --name servicebv1 \
    --namespace-id  ns-ngtw4luqvwo5pgnw \
    --dns-config "NamespaceId=ns-ngtw4luqvwo5pgnw,RoutingPolicy=WEIGHTED
,DnsRecords=[{Type=A,TTL=60}]"

Write a configuration file for service discovery. Note that you need to manually create a file called servicea. The arn in the file cannot be found directly on the console. It can only be spliced ​​manually through the id of the service

// serviceA-config.json
{
    "serviceRegistries": [
            {
                "registryArn": "arn:aws-cn:servicediscovery:cn-north-1:xxxxxx:service/srv-sqvht7gsx7ikq7um",
                "containerName": "simpleweb"
            }
        ]
}

Create a service and specify the service discovery configuration when creating the service

aws ecs create-service \
    --cluster workfargate \
    --service-name serviceA \
    --task-definition testservice \
    --desired-count 1 \
    --network-configuration "awsvpcConfiguration={subnets=[subnet-027025e9d9760acdd	],securityGroups=[sg-096df1a0cb9a6d7e9],assignPublicIp=ENABLED}" \
    --platform-version LATEST \
    --launch-type FARGATE \
    --cli-input-json file://serviceA-config.json

In the cloudmap console, check that the service has been registered successfully

Configure ecs service

Create a front-end service serviceA

Create cloudmap analysis

aws servicediscovery create-service \
    --name servicebv1 \
    --namespace-id  ns-ngtw4luqvwo5pgnw \
    --dns-config "NamespaceId=ns-ngtw4luqvwo5pgnw,RoutingPolicy=WEIGHTED
,DnsRecords=[{Type=A,TTL=60}]"

Specify service discovery configuration

// serviceA-config.json
{
    "serviceRegistries": [
            {
                "registryArn": "arn:aws-cn:servicediscovery:cn-north-1:xxxxxx:service/srv-sqvht7gsx7ikq7um",
                "containerName": "simpleweb"
            }
        ]
}

Create the front-end service serviceA, and specify the service discovery configuration when creating the service

Package testcurl image

FROM public.ecr.aws/docker/library/alpine:latest
RUN apk add --update curl && rm -rf /var/cache/apk/*
COPY ./entrypoint.sh .
RUN chmod +x entrypoint.sh
CMD ./entrypoint.sh
//./entrypoint.sh
while [ true ]; do
    sleep 3
    curl -v serviceb.ecs-mesh-local
done

serviceA uses the testcurl mirror

aws ecs create-service \
    --cluster workfargate \
    --service-name serviceA \
    --task-definition mesh-serviceA:8 \
    --desired-count 1 \
    --network-configuration "awsvpcConfiguration={subnets=[subnet-027025e9d9760acdd	],securityGroups=[sg-096df1a0cb9a6d7e9],assignPublicIp=ENABLED}" \
    --platform-version LATEST \
    --launch-type FARGATE \
    --cli-input-json file://serviceA-config.json

Create a backend service serviceBv1

Similarly, create a cloudmap analysis

aws servicediscovery create-service \
    --name servicebv1 \
    --namespace-id ns-ngtw4luqvwo5pgnw \
    --dns-config "NamespaceId=ns-ngtw4luqvwo5pgnw,RoutingPolicy=WEIGHTED
,DnsRecords=[{Type=A,TTL=60}]"

Specify service discovery configuration

// serviceBv1-config.json
{
    "serviceRegistries": [
            {
                "registryArn": "arn:aws-cn:servicediscovery:cn-north-1:xxxxxx:service/srv-rzhhkc5iwcmkkla5",
                "containerName": "simpleweb"
            }
        ]
}

Create a backend service serviceBv1, the backend service uses the zhaojiew/simpleweb mirror, and set the listening port through the environment variable MY_PORT

aws ecs create-service \
    --cluster workfargate \
    --service-name serviceBv1 \
    --task-definition mesh-serviceBv1:3 \
    --desired-count 1 \
    --network-configuration "awsvpcConfiguration={subnets=[subnet-027025e9d9760acdd	],securityGroups=[sg-096df1a0cb9a6d7e9],assignPublicIp=ENABLED}" \
    --platform-version LATEST \
    --launch-type FARGATE \
    --cli-input-json file://serviceBv1-config.json

Create a backend service serviceBv2

Similarly, create a cloudmap analysis

aws servicediscovery create-service \
    --name servicebv2 \
    --namespace-id  ns-ngtw4luqvwo5pgnw \
    --dns-config "NamespaceId=ns-ngtw4luqvwo5pgnw,RoutingPolicy=WEIGHTED
,DnsRecords=[{Type=A,TTL=60}]"

Specify service discovery configuration

// serviceBv2-config.json
{
    "serviceRegistries": [
            {
                "registryArn": "arn:aws-cn:servicediscovery:cn-north-1:xxxxxx:service/srv-ddh5y5tnqsmc3z5d",
                "containerName": "simpleweb"
            }
        ]
}

Create a backend service serviceBv2

aws ecs create-service \
    --cluster workfargate \
    --service-name serviceBv2 \
    --task-definition mesh-serviceBv2:2 \
    --desired-count 1 \
    --network-configuration "awsvpcConfiguration={subnets=[subnet-027025e9d9760acdd	],securityGroups=[sg-096df1a0cb9a6d7e9],assignPublicIp=ENABLED}" \
    --platform-version LATEST \
    --launch-type FARGATE \
    --cli-input-json file://serviceBv2-config.json

Now we have 3 services and expose port 80

resolve dns name

# nslookup servicebv1.ecs-mesh.local
Server:         172.31.0.2
Address:        172.31.0.2#53

Non-authoritative answer:
Name:   servicebv1.ecs-mesh.local
Address: 172.31.24.182

try to access

$ curl servicebv2.ecs-mesh.local:8090
success

Create and configure appmesh

https://docs.aws.amazon.com/zh_cn/AmazonECS/latest/developerguide/create-task-definition-classic.html

Through the above steps we have started the basic service framework.

Next, we need to connect the service to the grid, update the ecs task definition on the console, and enable the integrated appmesh

Configure error prompts, follow the prompts to configure

There is a pit here. The problem is that when creating a virtualnode in the mesh, the dns name must be set, and the dog will

After that, the injection can be successful, and the following information will be prompted

Afterwards, we need to update the task definition of each service separately and start a new task. It should be noted here that since we use the same task definition, we need to manually specify the virtualnode in the mesh every time. The principle can refer to the logic of ec2

After the agent is injected, the instance cannot be connected through exec. It is speculated that the agent may have processed the request. . . .

Modify the previous task definitions of serviceBv1 and v2, and use nginx for testing

Try to request backend v1 on ec2, it can be seen that envoy has taken over the request

$ curl servicebv1.ecs-mesh.local:8090
HTTP/1.1 200 OK
date: Tue, 28 Feb 2023 14:28:58 GMT
server: envoy
content-length: 7
content-type: text/html; charset=UTF-8
x-envoy-upstream-service-time: 1

Looking at the access log, it is expected that the dns name cannot be resolved. We manually create the resolution of serviceb in the cloudmap and point to any version of the backend. Similar to the ec2 environment, this resolution does not go through dns but is only forwarded by envoy

Manually register serviceb

After successfully accessing the service

Since the weight of 5050 is specified, the backend of both versions can receive the request

Change it to 100:0, and check that one of the services is no longer accepting new requests, indicating that our configuration is in effect

After that, we don't need to change any ecs applications, and we can complete complex configurations such as flow control only through appmesh

In fact, it is still difficult to operate on a client like ours. You can directly use ec2 instead. For mesh, the underlying application does not matter

This should be one of the most difficult tests in all articles so far, I hope everyone can gain something