← 2026 Work Logs

Work Logs

2026-01-25

Today I went for a haircut. When I came back, I found the system was unstable. It turned out that Jige had started two terminalid instances of the same service, which were preempting each other, causing a big problem.

  1. What I did today:

    1. Tried to migrate the cluster behind NAT, of course using the brand new legion to do this. My operations were as follows:

      • First, modified the kops cluster, created a new VPC using CIDR blocks 172.21.0.0/24 and 172.21.1.0/24. Then created a NAT for egress traffic.

        Originally planned to use a CIDR starting with 10.0, but after trying, AWS didn’t allow creating such a CIDR, so I switched to a CIDR starting with 172.21. There was a pitfall: I needed to point the existing load balancer in the cluster resource to the corresponding VPC (originally it was implicitly default, but now with an additional CIDR, I had to manually specify it).

      • Then created a new instance group pointing to the new VPC. There was a small hiccup: the new IG did not have S3 permissions, for some reason. After manually adding them, nodes joined the cluster normally.

      • Next step: manually migrate services to the new IG.

      • Finally, tear down the original IG.

      After completing all this, I found that the cluster’s egress traffic had only one IP, which caused some trouble for our IP rate-limiting service. I had to roll back; I must first unlock the http proxy skill before proceeding.

    2. The multi-agent was used to practice a script to automatically update the midas net asset value. Deepseek took quite a while to write it, but I felt quite satisfied with it. There is a core issue: if I didn’t notice an error in the early design, then I would face a huge waste of tokens and time, because I found that the agent doesn’t work very fast either.

      Currently, these coding agents are still quite primitive. They often exit or crash due to network issues during use. Getting them to perform serious long-running tasks still has somewhat poor SLI. This could also be an opportunity; upon simple reflection, it requires some knowledge of software engineering high availability to make it work.

  2. Thoughts: I have few thoughts today; they are all written inline in the above sections.

  3. What to do tomorrow?

    1. Design Yuan’s http proxy mechanism.
    2. After going live, re-migrate the cluster.