Notes

The Log: What every software engineer should know about real-time data’s unifying abstraction. Thoughts: What’s the difference between log/stream and messaging/notification service? Possibly that the former is for data flow whereas the latter for one time notifications between systems. Data silos in a company don’t help but, once connected, can unlock new insights and functionality. Logs seem to be a good way to do that. An interesting idea is that log enables putting data into both Hadoop-style batch and real-time (such as search indexing or monitoring) systems. (It may sound simple but can be complicated. For example, in a previous team, I tried aggregating a stream of affiliate order events and it wasn’t easy.) In the following image, what happens if you realize one of the upstream systems was producing bad data for, say, a few hours or days? How do you backfill data downstream and what do you do about the bad data inside the log?

Takeaways: I really liked the term “beg button”. Totally apt. Roads in the US are optimized for drivers, not pedestrians. However, I wonder if it weren’t, how bad would the traffic situation have been. For starters, public transportation would’ve to be lot better than what it is today. In US, city planners added attention-grabbers (for lack of a better term) that drivers can choose to ignore. So, onus is on pedestrians to keep themselves safe. (For example, who actually stops at stop signs? In my personal experience, hardly anyone.) In Netherlands, they planned for safety as part of infrastructure planning. Examples: The pedestrian crossing requires the walker to come down to the road level, cross and go back up. This implicitly means the road is naturally meant for cars. In Netherlands, however, the crossing remains at the same level and cars have to cross a bump - this also means they have to slow down. No “islands” in between 4-6 lane crossings, meaning pedestrians have to cross within a minute or so. Lanes narrow in Netherlands and that makes the drivers pay attention. In US, lanes are broad, so drivers usually drive faster than the speed limit. When I moved here from India, I thought lane crossing (especially the one on Mercer in Seattle) scene was almost perfect (because of what I was comparing it with). However, I now realize it can be better than this. Next up, I am interesting in reading a book called The Death and Life of Great American Cities.

I am doing something that I swore to not do a year or two ago: I am looking for new note taking apps. I thought I had sorted out everything after I made my notes public on my website. However, the note-capturing workflow on my iPhone is shitty (or, rather, non-existent). And that is something I’ve been finding crippling for quite some time now: I need the ability to write down my thoughts anywhere I am and that doesn’t work today. I’ve tried multiple apps on the phone and tried syncing content to my desktop. Nothing clicked. (Notion kind of worked but was terribly slow.) Features I care about: Longevity: thing should be around for awhile. Data should be completely under my control. (Don’t trust export etc. functionality from cloud-providers.) Hack-ability: at the minimum, I need an easy way to make all my notes public and, of course, on my own domain. (This is in line with what I do with my current notes - there is no point in keeping these things private.) Sync between devices. Anti-features: No reliance on plugins: software should work great out of the box. Top contenders so far: Logseq. (Most probably gonna settle on this one.) Obsidian: closed-source but looks good too.

Portland was fun. I randomly chose a hotel that had a fantastic location: There were a lot of highly rated restaurants, bars and coffee shops within couple of blocks of the hotel. Shopping and parking right across the street. Portland has good and cheap public transportation system. So, we were about to leave our car in parking whole Saturday. It was also our second trip there with the kiddo. The weather was great throughout the weekend, so lot of fun.

Data flow is: application -> cache -> database (i.e. cache sits in between the application and the database). There are 3 strategies: Write through: write to both in parallel. Write around: database only. Write back: cache only. It basically depends on whether, during a write, the application writes to the underlying database, cache or both.

Four Seasons Hotel in Buenos Aires. Posadas 1086, C1011 CABA, Argentina

435: Hwy 527 & 180th St SE - near Fusion India. Olive Way & 6th Ave 412: Good starting positions: Puget Park Dr & Silver Firs Dr - seems residential but 5 mins away. 132nd St SE & 35th Ave SE. Or Amazon Go. Olive Way & 6th Ave

Just add something like the following after the last (useful) command in the shell script: curl -fsS -m 10 --retry 5 -o /dev/null https://hc-ping.com/.../$? Documentation here.

My life is feeling haphazard right now. There are too many small things in my head - I also have a lot of time on my plate these days, because I’ve taken some time off work, and a lot of these small things are just fillers for the free time I have on hand. Here are a few things, in no particular order (of importance etc.): What book do I start now? Finish creating a trust. Find a new team at work. What more on self-hosting front? Start preparing for interviews. Finish home setup. Start working out regularly.

Here is an easy way to setup a Wireguard VPN server, Pi-hole & Unbound on a Raspberry Pi. Note: Set private-domain: "ketanvijayvargiya.com" in unbound.conf file and restart the container. (In Portainer, it was located in the /var/lib/docker/volumes/wireguard-pihole-unbound_unbound_config/_data directory.) version: "3.8" services: unbound: image: mvance/unbound-rpi:latest restart: unless-stopped volumes: - unbound_config:/opt/unbound/etc/unbound/ networks: wg-easy: ipv4_address: 10.8.1.4 wg-easy: depends_on: [unbound, pihole] environment: - WG_HOST=... - PASSWORD= - WG_DEFAULT_DNS=10.8.1.3 - WG_DEFAULT_ADDRESS=10.8.0.x image: weejewel/wg-easy volumes: - wg-easy:/etc/wireguard ports: - "51820:51820/udp" - "51821:51821/tcp" restart: unless-stopped cap_add: - NET_ADMIN - SYS_MODULE sysctls: - net.ipv4.ip_forward=1 - net.ipv4.conf.all.src_valid_mark=1 networks: wg-easy: ipv4_address: 10.8.1.2 pihole: depends_on: [unbound] image: pihole/pihole dns: - 127.0.0.1 environment: - TZ=America/Los_Angeles - WEBPASSWORD= - ServerIP=10.8.1.3 - PIHOLE_DNS_=10.8.1.4 - DNSSEC=true volumes: - pihole_config:/etc/pihole - pihole_dnsmasq:/etc/dnsmasq.d ports: - "9001:80/tcp" cap_add: - NET_ADMIN restart: unless-stopped networks: wg-easy: ipv4_address: 10.8.1.3 networks: wg-easy: ipam: config: - subnet: 10.8.1.0/24 volumes: pihole_config: pihole_dnsmasq: wg-easy: unbound_config: