Notes

I (finally) finished Oathbringer today. The book is so long, I dropped it a few months ago and decided I won’t read it again. Eventually, I picked it up recently and went through the last 12-15 hours of it. There was one part in the middle that bored me at the time. Some of the main characters were stuck in Shadesmar and nothing much was happening. This time, I got through it and feel glad I did. The part that stuck me the most is how accepting the mistakes of his past was the first step for Dalinar to become a better person. Second, he says the next step is always the most important one that a person can take. In true Sanderson style, the characters went through their own struggles and, in some cases, found ways to overcome them. For e.g., Szeth decided to follow a man he once tried to assassinate, Teft overcame his addiction to become a Knight Radiant, Shallan eventually decided that she loved Adolin and not Kaladin, Rock broke his vow - to never kill someone - to save Kaladin etc. I am sure I’ll remember more examples if I press my brain. I have the next book in my Audible library but it too is long (at 57+ hours!), so I’ll start it after some time. I also saw somewhere that the one after the next is expected to come out next year, so there is a lot of ground to cover in the series!

I am trying to integrate Netlify CMS on this website. This post is a test for that. Random image for the test: Edit, on Dec 7 20222: I migrated from Netlify’s identity management to GitLab’s Client-Side PKCE Authorization. I’ll just save this edit and verify that I am able to push the change to the my GitLab repo.

I watched No Time To Die today. IIRC Bond fell in love for the 3rd time (with a daughter, no less): before this, he got married once and his wife died the day on their wedding, the other time was with Vesper in Casino Royale who kind of betrayed him and died as well. In this movie, his family survived but Bond died. The movie reminded me of Indiana Jones: bullets were flying past Bond but each of his shots were on point.

I learned a few random things lately: India doesn’t have a big number of skyscrapers despite high population because of two reasons. One, albeit minor, is that skyscrapers require consistent power supply and that’s hard in India. Two, the bigger one, is the government allowed FSI: a ratio of total floor space within the building to the area of land on which the building is built. FSI varies with project or location but is usually small (2-5) in India compared to that in other countries where it can go as high as 25. Third place: The YouTube presenter argues that a third place, other than your home and office, something like a local pub etc. where you can walk to, is socially important. Allows you to serendipitously bump into a set of people periodically. That’s where new community members can form first friends. The following don’t serve the same purpose: Self-organized backyard parties because you already know the people that you’ll meet. Malls or those pubs that you need to drive to because you possibly won’t bump into the same people multiple times. (Also, I myself can’t imagine going up to strangers in a random bar and starting a conversation. ) Recently, as I’ve started going to the office every week or two, I’ve noticed how nice random conversations are. They just don’t happen on video calls when you are working from home because you only call people with an agenda.

In other news, we got a TV today: it has a gorgeous 65 inch OLED screen. I am kind of tired of watching videos on my phone or laptop, so I spent at least 4 hours on the TV! Part of it went into watching Dune.

Today, I was reminded of a book that I had heard about earlier: how to read a book. I was about to borrow it from the library when I realized that it’ll become yet another thing on my todo plate, however useful it might turn out to be. So, I didn’t check out. I do this often but here’s another list of what’s, or should be, on my plate right now: Entertainment: Oathbringer. YouTube and other stuff on TV. Books: CI. If possible, some stuff on distributed systems. As usual, work, workout and household chores. When I look back on my notes from a week or two before, I see pretty much the same things there. I think what’s happening is I am dividing my time disproportionately in these buckets. For example, a lot of it on self-hosting and randomly checking for Black Friday deals. Life will become pretty simple if I stuck to such a small number of things.

Multi-region Kafka: The blog describes an architecture where: Stream producers only write to a stream in their own region. There is another stream - “aggregate stream/cluster” - which is replicated across all the regions. (In other words, data is duplicated in every region.) Data from the region-level streams is moved into this one. Consumers need to be more complex if they want to support failovers. They work in 2 ways: Active-active. Each region has an active consumer that runs redundant business logic. If the system overall needs to create one view of data - for e.g., the blog suggests one database, across all regions, that contains surge pricing data - it needs a coordinator that tells all the region-level consumers which region contains the database they need to write to. If the region has issues, it coordinates the failover to another region. I found a couple of things weird in the article. Why did they create a single database for surge pricing? If the stream data was anyway replicated across all the regions, they could’ve created a region-level database and have the database clients pick the region closest to them. It says the Flink job is resource-intensive. So, isn’t it expensive to duplicate that job across all their regions? Active-passive: There is one active consumer in one region. However, how do you support failover to another region? They replicate checkpoints across the regions. That way, a new consumer can restart from the last known checkpoint. One caveat is around offsets. (I think this is same as stream-sequence-ids.) These offsets can can be in different order in different regions of the aggregation stream (for e.g., writes from a regional stream of my region will come in faster than those from other regions). So, they describe an algorithm to maintain offset mappings across regions so that a new consumer can pick the right offset during failover. (I didn’t read this algorithm in detail.)

I am looking for an easy way to share these Obsidian notes publicly on my website. I just don’t want to learn something completely new for that. But I also don’t want to spend too much time setting something up.

I have a lot of time on my hands these days because my family is traveling to India. As usual, a lot of that goes unaccounted for! Work is light. I do have a big project on hand but progress is slow because folks, that I need to talk to to progress, are busy with AWS’s Reinvent conference. I joined my current team recently, so don’t have any deliverables for the conference. If I had, life would have been pretty hectic right now. I spent some time on improving my home server setup. The biggest driver was how one service that I depended on a lot - Tailscale, to access my home services outside my home network - ate my phone battery crazy, something like 20-25% on a 10 day average. I tried replacing Tailscale with Cloudflare Zero Trust. It worked but then I realized some problems, after running it for couple of weeks, so went searching again. It too used 7-9% battery on average. It would see all our internet traffic, so not a great choice privacy-wise. It only uses its own DNS and doesn’t support Pihole level ad blocking customizations. I couldn’t even use the Pihole instance I have running at my home. All the setup was UI-heavy and their UI had too many options. I prefer IAAC, i.e. infrastructure as code, and, while that was possible through Terraform, I’d have to learn a whole new thing and how to move my existing resources into the config. I realized that running a background app all the time will eat battery, no matter what. The app could also improve or degrade over time, outside my control. So, I figured the best course of action would be to expose my services to the public internet, scary though it may be. I then put an authentication service between the internet and my services - it is called Authelia. Its basic use case is to add authentication in front of applications that don’t have any. But I thought of making it sit in front of all my self-hosted applications by default, so that it becomes one place where I need to ensure strong passwords etc. and let it handle bruce-force password attacks (for e.g., it blocks callers for X minutes if they fail Y attempts). I also setup DNS over TCP on my home server. Earlier, I used to rely on Tailscale to relay DNS queries from my phone to Pihole on my home server when I was away from home. However, as part of moving away from Tailscale, I exposed Pihole over DoT. Turned out to be a fun exercise. I couldn’t do regular DNS over port 53 or whatever because that has security implications. I watched a TV show called Bodyguard. I chose it because it was a mini-series, so no long term time commitment.

One way to access pihole outside of home would be DoH/DoT on a publicly exposed port and have Traefik sit in the middle. It has its concerns though: This won’t support any authentication, so (theoretically) anyone in the world will be able to use this. I can obfuscate by using a random string as URL but still. Latencies will be high further away from home I am. Need to figure out how to generate mobile configs. Still, the idea sounds intriguing. Helpful resources just in case I want to revisit this in future: https://hackmd.io/@gushmazuko/SJdUS6gSF https://www.reddit.com/r/pihole/comments/fy8zmu/pihole_traefik_dnsovertls/