Notes

Motivation It has been quite awhile since I’ve been trying to figure out how I want to operate at work. The company expects me to be more of a tech lead because that will allow my manager to scale out and get work out of a larger workforce. And while that does have some merits, it isn’t the best way for me. It isn’t fun and I don’t get to learn much. It isn’t what other companies expect from SDEs, so it decreases my employability. (For example, Facebook and possibly both Google and Microsoft expect their senior engineers to contribute a lot of code). Even within Amazon, they’ll look at my deliverables at the end of the year. If I only work through others, I won’t have a lot of tangible stuff to show forth. I’ve also heard from my manager that whatever I do inside the team isn’t going to take me to the next level, so why do that as much? The other side of the story is that not behaving as much of a tech lead could lead to some reputational loss, but I think that’s an acceptable tradeoff. ...

The following will query CloudWatch Logs Insights for last 24 hours. If you want to search for a different duration, adjust the --start-time and --end-time parameters accordingly. queryId=$(aws logs start-query \ --log-group-name '<log-group-name>' \ --start-time `date -j -v-1d +%s` \ --end-time `date +%s` \ --query-string ' fields @timestamp, @message | sort @timestamp desc | limit 20 ' | jq --raw-output '.queryId' ) aws logs get-query-results \ --query-id $queryId

There are two ways of doing this and they both work with Git submodules. git reset --hard <commit-ID> Obviously risky as it will overwrite all local changes in the working directory. Works better when you only depend on Git submodules but don’t make any code changes to them (such as Hugo themes). Or: git checkout <commit-ID>

Pytorch: x += 1 and x = x + 1 are not equivalent in Pytorch! The former is in-place and the latter is not, so be careful. More explanation is available here: x = x + 1 is not in-place, because it takes the objects pointed to by x, creates a new Variable, adds 1 to x putting the result in the new Variable, and overwrites the object referenced by x to point to the new var. There are no in-place modifications, you only change Python references (you can check that id(x) is different before and after that line). ...

We are expecting our first baby soon, sometime in mid November! When I learnt about this, it took awhile for the feeling to sink in. I am obviously super-excited and looking forward to when the kiddo arrives. I am also intimidated by the unknowns: I’ve heard some rumors but will only understand them when the time comes.

Step Function emits the following execution metrics for each state machine: ExecutionsSucceeded. ExecutionsAborted. ExecutionsFailed. ExecutionsTimedOut. We can get the overall error rate for a given state machine by doing the following: aws cloudwatch get-metric-data \ --region us-east-1 \ --start-time 2020-06-01T00:00:00Z \ --end-time 2020-08-01T01:00:00Z \ --scan-by TimestampAscending \ --metric-data-queries ' [ { "Id": "executionsAborted1", "MetricStat": { "Metric": { "Namespace": "AWS/States", "MetricName": "ExecutionsAborted", "Dimensions": [ { "Name": "StateMachineArn", "Value": "<state-machine-ARN>" } ] }, "Period": 300, "Stat": "Sum" }, "ReturnData": false }, { "Id": "executionsFailed1", "MetricStat": { "Metric": { "Namespace": "AWS/States", "MetricName": "ExecutionsFailed", "Dimensions": [ { "Name": "StateMachineArn", "Value": "<state-machine-ARN>" } ] }, "Period": 300, "Stat": "Sum" }, "ReturnData": false }, { "Id": "executionsSucceeded1", "MetricStat": { "Metric": { "Namespace": "AWS/States", "MetricName": "ExecutionsSucceeded", "Dimensions": [ { "Name": "StateMachineArn", "Value": "<state-machine-ARN>" } ] }, "Period": 300, "Stat": "Sum" }, "ReturnData": false }, { "Id": "executionsTimedOut1", "MetricStat": { "Metric": { "Namespace": "AWS/States", "MetricName": "ExecutionsTimedOut", "Dimensions": [ { "Name": "StateMachineArn", "Value": "<state-machine-ARN>" } ] }, "Period": 300, "Stat": "Sum" }, "ReturnData": false }, { "Id": "errorRate", "Expression": "1 - (executionsSucceeded1)/(executionsSucceeded1 + executionsTimedOut1 + executionsFailed1 + executionsAborted1)" } ] '

Top-level error rate: aws cloudwatch get-metric-data \ --region us-east-1 \ --start-time 2020-08-01T00:00:00Z \ --end-time 2020-08-02T00:00:00Z \ --scan-by TimestampAscending \ --metric-data-queries ' [ { "Id": "m1", "MetricStat": { "Metric": { "Namespace": "AWS/ApiGateway", "MetricName": "5XXError", "Dimensions": [ { "Name": "ApiName", "Value": "<service-name>" }, { "Name": "Stage", "Value": "<stage-name>" } ] }, "Period": 300, "Stat": "Average" } } ] ' Or, for API-level, just add relevant dimensions: ...

List error-metrics for an AWS API Gateway service: aws cloudwatch list-metrics \ --region us-east-1 \ --namespace AWS/ApiGateway \ --metric-name 5XXError \ --dimensions ' [ { "Name": "ApiName", "Value": "<service-name>" } ] ' Or all metrics for AWS Step Functions: aws cloudwatch list-metrics \ --region us-east-1 \ --namespace AWS/States

Create aws kms create-grant \ --region us-east-1 \ --key-id <key-ARN> \ --grantee-principal <grantee-principal-role-ARN> \ --operations Encrypt Decrypt GenerateDataKey \ --constraints EncryptionContextSubset={'foo'='bar'} Note: The list of valid --operations can be found here. If you don’t want to supply encryption context, just omit the --constraints parameter. List aws kms list-grants \ --region us-east-1 \ --key-id <key-ARN> Retire aws kms retire-grant \ --region us-east-1 \ --grant-id <grant-id> \ --key-id <key-ARN>

export AWS_DEFAULT_REGION=us-west-2