Skip to content

Instantly share code, notes, and snippets.

@palewire
palewire / athena.py
Last active April 2, 2024 23:31
Python helpers for running Amazon Athena queries
"""Utilities for working Amazon Athena.
Example:
To run a query and get the results as a pandas DataFrame:
>>> query = "SELECT * FROM my_table"
>>> df = get_df_from_athena(query)
>>> df.head()
@palewire
palewire / README.md
Last active December 18, 2022 10:39
Install The Tor Project's snowflake-proxy on a Raspberry Pi

Your Raspberry Pi can join The Tor Project network that helps Russians read censored news sites, including Twitter.

These are all the commands necessary to spin up a Snowflake proxy that gives Tor users a way around government attempts to block access.

You can also run a Snowflake proxy from your web browser with Firefox or Chrome. Standing it up on your Raspberry Pi is a way to support the system 24 hours a day. And, unlike other Tor server setups, it doesn't require a static IP address.

@palewire
palewire / numoji.py
Last active March 14, 2022 11:51
numoji.py
def numoji(number):
"""Convert a number into a series of emojis.
Args:
number (int): The number to convert into emoji
Returns: Am emoji string
"""
# Convert the provided number to a string
s = str(number)
@palewire
palewire / README.md
Last active February 28, 2022 17:51
U.S. Earthquake Risk in 3D

U.S. Earthquake Risk in 3D

[
{
"case_number":"2017-04514",
"slug":"eddie-rosendo-lino",
"first_name":"Eddie",
"middle_name":"Rosendo",
"last_name":"Lino",
"death_date":"2017-06-18T00:00:00.000Z",
"death_year":2017,
"age":23.0,
@palewire
palewire / README.md
Last active May 24, 2023 18:09
How to deploy a Prefect agent to Google Kubernetes Engine

How to deploy a Prefect agent to Google Kubernetes Engine

This post contains code and commands you can use to deploy Prefect agents to Google Cloud’s Google Kubernetes Engine. The agents stand ready to execute workflows triggered by Prefect projects. One agent can run tasks from multiple projects.

The example here demonstrates how to create a single agent with minimal customization. It is configured with a Dockerfile, which installs necessary dependencies, and a k8s.cfg file, which connects the system to a Prefect account.

Agents are deployed via the gcloud command-line utility and its kubectl extension. Proper permissions within a project on Google Cloud are required.

Getting started

@palewire
palewire / README.md
Last active April 15, 2024 20:38
How to push tagged Docker releases to Google Artifact Registry with a GitHub Action

How to push tagged Docker releases to Google Artifact Registry with a GitHub Action

Here's how I configured a GitHub Action so that a new version issued by GitHub's release interface will build a Dockerfile, tag it with the version number and upload it to Google Artifact Registry.

Before you attempt the steps below, you need the following:

  • A GitHub repository that contains a working Dockerfile
  • The Google Cloud SDK tool gcloud installed and authenticated

Create a Workload Identity Federation

@palewire
palewire / README.md
Last active January 5, 2024 09:49
How to open Excel files in Python without pandas

How to open Excel files in Python without pandas

The pandas data-analysis tool can do almost everything. And because it can do almost everything, it's a massive hunk of code. So you might not want to include it in your project if you can avoid taking on the weight.

One of pandas' handiest features is quickly opening Excel spreadsheets. If that's all you need, here's how to accomplish the task with openpyxl, a much lighter dependency.

@palewire
palewire / run.py
Last active January 19, 2023 21:48
A command-line interface for running Jupyter Notebooks.
# -*- coding: utf-8 -*-
"""
A command-line interface for running Jupyter Notebooks.
Usage: run.py [OPTIONS] [NOTEBOOK_PATHS]...
Executes Jupyter Notebooks from the command line.
Expects one or more file paths input as arguments.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.