Notes

I began part 1 of the Fastai course sometime in Q4 2019. After an inordinate amount of time for something that I could and should have finished in about 3-4 months, I git pushed my last notebook early last week. Nevertheless, I was excited but then Fastai launched a newer, and possibly a more improved, version of their course a couple of days later! The new course is apparently better because it incorporates some aspects of traditional Machine Learning (as opposed to how the previous one was only about Deep Learning) and is taught with a newer version of the Fastai library that they launched alongside the course. ...

Ebey’s Landing National Historical Reserve: looks beautiful. Bluff trail (3.5 miles). Whidbey Island Bagel Factory: it was pretty good. There is one beach on each side of the ferry terminal - can visit those next time. Not do: Fort Casey’s: I’ve already been there twice, so try to find something else next time!

Motivation It has been quite awhile since I’ve been trying to figure out how I want to operate at work. The company expects me to be more of a tech lead because that will allow my manager to scale out and get work out of a larger workforce. And while that does have some merits, it isn’t the best way for me. It isn’t fun and I don’t get to learn much. It isn’t what other companies expect from SDEs, so it decreases my employability. (For example, Facebook and possibly both Google and Microsoft expect their senior engineers to contribute a lot of code). Even within Amazon, they’ll look at my deliverables at the end of the year. If I only work through others, I won’t have a lot of tangible stuff to show forth. I’ve also heard from my manager that whatever I do inside the team isn’t going to take me to the next level, so why do that as much? The other side of the story is that not behaving as much of a tech lead could lead to some reputational loss, but I think that’s an acceptable tradeoff. ...

The following will query CloudWatch Logs Insights for last 24 hours. If you want to search for a different duration, adjust the --start-time and --end-time parameters accordingly. queryId=$(aws logs start-query \ --log-group-name '<log-group-name>' \ --start-time `date -j -v-1d +%s` \ --end-time `date +%s` \ --query-string ' fields @timestamp, @message | sort @timestamp desc | limit 20 ' | jq --raw-output '.queryId' ) aws logs get-query-results \ --query-id $queryId

There are two ways of doing this and they both work with Git submodules. git reset --hard <commit-ID> Obviously risky as it will overwrite all local changes in the working directory. Works better when you only depend on Git submodules but don’t make any code changes to them (such as Hugo themes). Or: git checkout <commit-ID>

Pytorch: x += 1 and x = x + 1 are not equivalent in Pytorch! The former is in-place and the latter is not, so be careful. More explanation is available here: x = x + 1 is not in-place, because it takes the objects pointed to by x, creates a new Variable, adds 1 to x putting the result in the new Variable, and overwrites the object referenced by x to point to the new var. There are no in-place modifications, you only change Python references (you can check that id(x) is different before and after that line). ...

We are expecting our first baby soon, sometime in mid November! When I learnt about this, it took awhile for the feeling to sink in. I am obviously super-excited and looking forward to when the kiddo arrives. I am also intimidated by the unknowns: I’ve heard some rumors but will only understand them when the time comes.

Step Function emits the following execution metrics for each state machine: ExecutionsSucceeded. ExecutionsAborted. ExecutionsFailed. ExecutionsTimedOut. We can get the overall error rate for a given state machine by doing the following: aws cloudwatch get-metric-data \ --region us-east-1 \ --start-time 2020-06-01T00:00:00Z \ --end-time 2020-08-01T01:00:00Z \ --scan-by TimestampAscending \ --metric-data-queries ' [ { "Id": "executionsAborted1", "MetricStat": { "Metric": { "Namespace": "AWS/States", "MetricName": "ExecutionsAborted", "Dimensions": [ { "Name": "StateMachineArn", "Value": "<state-machine-ARN>" } ] }, "Period": 300, "Stat": "Sum" }, "ReturnData": false }, { "Id": "executionsFailed1", "MetricStat": { "Metric": { "Namespace": "AWS/States", "MetricName": "ExecutionsFailed", "Dimensions": [ { "Name": "StateMachineArn", "Value": "<state-machine-ARN>" } ] }, "Period": 300, "Stat": "Sum" }, "ReturnData": false }, { "Id": "executionsSucceeded1", "MetricStat": { "Metric": { "Namespace": "AWS/States", "MetricName": "ExecutionsSucceeded", "Dimensions": [ { "Name": "StateMachineArn", "Value": "<state-machine-ARN>" } ] }, "Period": 300, "Stat": "Sum" }, "ReturnData": false }, { "Id": "executionsTimedOut1", "MetricStat": { "Metric": { "Namespace": "AWS/States", "MetricName": "ExecutionsTimedOut", "Dimensions": [ { "Name": "StateMachineArn", "Value": "<state-machine-ARN>" } ] }, "Period": 300, "Stat": "Sum" }, "ReturnData": false }, { "Id": "errorRate", "Expression": "1 - (executionsSucceeded1)/(executionsSucceeded1 + executionsTimedOut1 + executionsFailed1 + executionsAborted1)" } ] '

Top-level error rate: aws cloudwatch get-metric-data \ --region us-east-1 \ --start-time 2020-08-01T00:00:00Z \ --end-time 2020-08-02T00:00:00Z \ --scan-by TimestampAscending \ --metric-data-queries ' [ { "Id": "m1", "MetricStat": { "Metric": { "Namespace": "AWS/ApiGateway", "MetricName": "5XXError", "Dimensions": [ { "Name": "ApiName", "Value": "<service-name>" }, { "Name": "Stage", "Value": "<stage-name>" } ] }, "Period": 300, "Stat": "Average" } } ] ' Or, for API-level, just add relevant dimensions: ...

List error-metrics for an AWS API Gateway service: aws cloudwatch list-metrics \ --region us-east-1 \ --namespace AWS/ApiGateway \ --metric-name 5XXError \ --dimensions ' [ { "Name": "ApiName", "Value": "<service-name>" } ] ' Or all metrics for AWS Step Functions: aws cloudwatch list-metrics \ --region us-east-1 \ --namespace AWS/States