Skip to content

Instantly share code, notes, and snippets.

View softwaredoug's full-sized avatar
😇

Doug Turnbull softwaredoug

😇
View GitHub Profile
@softwaredoug
softwaredoug / capital_one_transfer.txt
Created November 1, 2023 13:15
edit capital one recurring transfer
How to edit a recurring transfer in capital one
Find an occurrence of that transfer in an account's transaction history, and you'll see an "EDIT" button
You're welcome
@softwaredoug
softwaredoug / parse_stack.py
Last active January 3, 2024 22:50
VisualVM Forward Calls Stacktrace to Flamegraph format
"""Parses a VisualVM Forward Call Tree CSV file and outputs a flamegraph.
In VisualVM CPU Snapshot, select "Export", and choose "Forward Call Tree" and save
as CSV. The input will look something like:
"org.apache.spark.util.collection.ExternalSorter.writePartitionedFile(ExternalSorter.java:388)", "1,252 ms (100.0%)"
" Self time", "1 ms (0.01%)"
" org.apache.spark.util.collection.ExternalSorter.someOtherFunction(ExternalSorter.java:388)", "1,150 ms (95.0%)"
" Self time", "75 ms (5.0%)"
(Other calls from of someOtherFunction)
@softwaredoug
softwaredoug / benchmark.py
Created August 15, 2023 14:46
Numpy bitcount benchmark
import timeit
import numpy as np
# Naive bitcount implementation without popcount
m1 = np.int64(0x5555555555555555)
m2 = np.int64(0x3333333333333333)
m3 = np.int64(0x0F0F0F0F0F0F0F0F)
m4 = np.int64(0x0101010101010101)
mask = np.int64(-1)
# Use a neovim session per working directory
#!/bin/bash
(which nvr > /dev/null) || pip3 install neovim-remote
working_dir_slug() {
CWD=$(pwd)
CWD=${CWD//\//_}
echo $CWD
}
@softwaredoug
softwaredoug / docker-compose-script
Created May 19, 2023 12:45
Put as 'docker-compose' and it will protect you from inadvertent `docker-compose down`
#!/bin/bash
args=($@)
for i in "${!args[@]}"; do
arg=${args[$i]}
if [[ "$arg" = "down" ]];then
echo "Stopping instead of downing"
echo "Use 'docker-compose destroy' to destroy"
args[i]="stop"
echo "Running docker-compose ${args[@]}"
elif [[ "$arg" = "destroy" ]];then
@softwaredoug
softwaredoug / instead_of_slack.md
Last active January 24, 2023 16:30
What to do instead of a slack message

Slack messages are annoying, mostly go unread. Slack has no way of prioritizing or organizing itself. I've become more and more disaffected by Slack over time as a way of doing work. Instead, I prefer these other ways of async work, where it's tracked more permanently, and the tooling is built for a certain type of interaction. I view slack as work social media, a kind of ephemeral place or community. A place, at most, to share an idea with no commitment to get a response.

I don't expect Slack to work well when...

  • I have a half-formed thought / brainstorm, and I want to "get it down", and need to "work it out". Don't put it on slack, that'll distract everyone, and probably be misunderstood anyway.
    • Put it in your notetaking system (like logseq) until its fully formed, then get it in a google doc for more feedback
  • I have a more fully-formed thought / idea / brainstorm and wish to share it with the team or an individual
  • Write a short google doc and put that in slack asking for f
@softwaredoug
softwaredoug / encode-wikipedia-sentences.py
Created January 1, 2023 20:51
Encoding wikipedia sentences with sentence encoder
import numpy as np
import os
from time import perf_counter
from sentence_transformers import SentenceTransformer, LoggingHandler
import logging
logging.basicConfig(format='%(asctime)s - %(message)s',
datefmt='%Y-%m-%d %H:%M:%S',
@softwaredoug
softwaredoug / defaults.py
Last active January 23, 2022 22:56
Python default parameters work in unexpected ways. Here we allow a per-call default, where objects are constructed on call, not function creation.
from inspect import signature
import random
from copy import deepcopy
def argue(fn):
"""Copies defaults _per call_ to give more expected python default parameter functionality.
Usage
-----
@softwaredoug
softwaredoug / i-dont-like-nbdev.md
Last active December 29, 2023 14:32
These are just notes from learning nbdev, that may turn out to be false, and I encourage that feedback.

I am working on a project contemplating the best use of notebooks in our search relevance workflow. We're a cross-disciplinary team of software engineers and data scientists. Recently, to decide best practices, I watched the two famous talks I don't like Notebooks by Joel Grus and I like notebooks by nbdev creator Jeremy Howard. As a senior dev, I want to have opinions for how my team should develop both the notebooks and any underlying libraries.

Positive things about nbdev and notebooks

  • Writing docs leads to better code - I have written better code when I know its being consumed as documentation by others, and needs to be read. I fully agree with the amazing feedback loop between writing and coding that creates much better libraries
  • Jupyter as a dev env - For some people, Jupyter is their preferred dev environment, and should be supported as such.
  • Philosophy - I generally agree with the philo
@softwaredoug
softwaredoug / TimestampPolicy.java
Created November 11, 2020 15:32
timestamp policy for Kafka
new CustomTimestampPolicyWithLimitedDelay<Long, SearchQueryEvent>(
new GetSearchQueryTimestampFunction(),
Duration.standardMinutes(5),
previousWatermark
);