Notes

I am one of those people who function better by writing things down. One day, I realized that most of my notes don’t have to be private, so here they are - my second brain. Be warned that, if you stumble upon something here that doesn’t make sense to you, it isn’t meant to!

Today I learned

Deep Learning: Whenever you define a neural network, either as a vanilla Python function or as a nn.Module, it should only take a mini-batch of training data as input. For e.g., for a net that works on images, the input will only be X whose shape could be [256, 1, 28, 28]. Here, 256 is the number of items in each mini-batch, 1 is the number of channels (for gray-scale images) and 28 * 28 is the size of the images. First element of the input is always the batch-size. Loss function should only return a scalar (or a Tensor of size 1). That is because PyTorch’s backward function only works on scalars.

Seattle - things to do outdoors

Hikes Mailbox - difficult, so attempt when there isn’t snow. https://www.kingcounty.gov/depts/transportation/metro/schedules-maps/trailhead-mailbox-peak.aspx North Cascades Hidden Lake Lookout - 8 mi. Alltrails. Perry Creek - 10.5 mi. Olympic Lena Lake - looks gorgeous for a summer hike - 7 mi. Google maps. https://www.alltrails.com/trail/us/washington/colchuck-lake-trail - 8 miles. Lime Kiln Trail - 7 miles. Only one hour from Seattle. Cape Lookout - 5 miles - 4.5 hours drive from Seattle. Longer through US-101 but might be prettier. Good camping locations Moran State Park on San Juan Islands. (Has five freshwater lakes for swimming and non-motorized boating.) Spencer Spit State Park. Backpacking Enchantments. (It’s difficult.) Hoh River Trail. Jade Lake: ~26 miles over 3 days with elevation gain, which means a lot of hiking in addition to learning how to backpack. (Probably not that great for me.) Resources: ...

Where to relocate from United States?

I am considering relocating to London or Vancouver BC next year. Differences between the two places Major: London is very close to India. For e.g., Delhi is just 9 hours away as compared to ~22 hours from Vancouver. We’ll be upper middle class in Vancouver and middle class in London. We won’t starve or anything, just that we’ll have lesser savings in the latter. Salaries will be lower in London. Tech scene less bustling in London, so lesser opportunities to switch companies. Taxes, rents and expenses will most probably be higher in London. While we could buy a home in Vancouver right when we move, we won’t be able to in London for at least a few years. Minor: London’s outdoor scene isn’t as great as that in Vancouver. Should be okay though as Himalayas and Alps are still accessible with a little bit of planning. On the other hand, London is very close to Europe (including Norway and Iceland)! Public transportation better in London. City vibe better in London. It will take slightly longer to get permanent residency in London (~5 vs <2 years). What’s the same Healthcare is free. Education until high school: definitely free in London but probably so in Vancouver too. Other noteworthy things London: Visa situation for parents: While I am on work visa, they can apply for a long-term Standard Visitor visa that lasts 2, 5 or 10 years. That will allow them to stay for 6 months on every visit. Once I become a UK citizien, they can get a family visa which will allow them to stay for a longer duration and make them eligible for UK healthcare. College isn’t free in the UK but could be in some European countries. Vancouver BC Where to live: https://www.teamblind.com/post/Good-places-for-immigrants-to-settle-in-Greater-Vancouver-2hGoqL4c

Fastai vs Dive into Deep Learning, to learn Deep Learning

What I learnt about moving to London from an Amazonian

Teams in: London. AWS Code Guru. Prime Video (i.e. PV) has a huge presence. Alexa. A new team from IMDb TV, possibly because they have some technology overlap with PV. Cambridge. S3. Kinesis. His team, CDK, is geographically distributed, so he reports to a manager in Seattle. Visa and relocation: Amazon relocates on some tier 2 general work visa. Monthly quota (as opposed to yearly quota in H1B). Took him 2.5 months, even after delays. Temporary accommodation for 1 month. Stay: City Center, which is where Amazon is. (No downtown.) Amazon in zone 1. Buying a home: Zone 1 crazy expensive. Zone 2 & 3 - apartment maybe. Zone 3 onwards - houses maybe. Zone 4 to office - 45 mins one way through subway. Rent: Some US-style apartments, but mostly condos rented out by owners. Managed either by owners or, more commonly, by agencies. Horror stories common. If landloards specific about how tenant should maintain the place, that’s a sign that the place might be good. ~2000 pounds for 2BR in zone 2. No car needed. Good public transportation. Finances: 30-50% salary drop. Significant increase in tax. No car expenses. Life: Cultural diversity is insanely good. Outdoors not as accesible (because you’ll probably take trains which will take time) or as beautiful as that in PNW. Flights to Europe dirt cheap.

Syncthing

Install Linux VERSION=1.8.0 WORKING_DIR=/tmp/syncthing-$VERSION rm -rf $WORKING_DIR mkdir $WORKING_DIR cd $WORKING_DIR wget https://github.com/syncthing/syncthing/releases/download/v$VERSION/syncthing-linux-amd64-v$VERSION.tar.gz tar xzvf syncthing-linux-amd64-v$VERSION.tar.gz cd syncthing-linux-amd64-v$VERSION # If the following command throws an error like: "cp: cannot create regular file `/usr/local/bin/syncthing': Text file busy", # shutdown the current Syncthing session from the UI and retry. sudo cp syncthing /usr/local/bin Note: Change version number to the latest one. Mac brew cask install syncthing If you run into any issues (for e.g., Syncthing doesn’t start), try starting the daemon through the command line to see what the problem is: ...

Today I learned

Statistics While the following from statistics is too fundamental to be interesting, I struggled (and failed) to fully grasp it a few years ago. Hopefully, it will stick this time! Anyway, before I forget: Standard deviation is the square root of variance. Variance is the average of squared sum of differences between mean and observed values. Standard deviation is a measure of how spread out observed values are. Normal or Gaussian distribution is simply a bell curve, something that is fairly common in the real world. For such a distribution, 68% of the values lie within 1 standard deviation of the mean, 95% within 2 and 99.7% within 3. Standard normal disribution: is a (special) normal distribution which has a mean of 0 and standard deviation of 1. The y-axis of such a graph is just numbers but the x-axis is called the Z-score. You can convert a normal distribution to a standard normal distribution by calculating the Z-score as: ((observerd-value - mean) / standard-deviation). The area under the curve is 1. If you search online, you’ll find a table that, for a given Z-score, gives the area of the curve on the left of that score. That will answer things like, what’s the probability that a given value X is less than something. Resources: ...

Today I learned

Fastai: Initially, when you start training, accuracy continues to go up. However, the model will start memorizing after sometime and validation set accuracy will decrease. That is when the model starts to overfit. The model looks at all images exactly once in every epoch. Architecture (for e.g., ResNet) is a template for a mathematical function. When trained, it turns into a model which contains parameters, possibly in millions, appropriate to tackle the problem at hand. Train = fit.

What next, after part 1 of the Fastai 2019 course?

I began part 1 of the Fastai course sometime in Q4 2019. After an inordinate amount of time for something that I could and should have finished in about 3-4 months, I git pushed my last notebook early last week. Nevertheless, I was excited but then Fastai launched a newer, and possibly a more improved, version of their course a couple of days later! The new course is apparently better because it incorporates some aspects of traditional Machine Learning (as opposed to how the previous one was only about Deep Learning) and is taught with a newer version of the Fastai library that they launched alongside the course. ...

Whidbey Island - things to do

Ebey’s Landing National Historical Reserve: looks beautiful. Bluff trail (3.5 miles). Whidbey Island Bagel Factory: it was pretty good. There is one beach on each side of the ferry terminal - can visit those next time. Not do: Fort Casey’s: I’ve already been there twice, so try to find something else next time!