Marck Vaisman wahalulu

App Install Plan

Critical

iTerm2
- https://iterm2.com/
Command Line Tools
- xcode-select –install

R to python data wrangling snippets

The dplyr package in R makes data wrangling significantly easier. The beauty of dplyr is that, by design, the options available are limited. Specifically, a set of key verbs form the core of the package. Using these verbs you can solve a wide range of data problems effectively in a shorter timeframe. Whilse transitioning to Python I have greatly missed the ease with which I can think through and solve problems using dplyr in R. The purpose of this document is to demonstrate how to execute the key dplyr verbs when manipulating data using Python (with the pandas package).

dplyr is organised around six key verbs:

Create a local Vagrant base box from an existing VirtualBox VM

What

Setting up a new development VM should be as easy as 2 commands. And it is:
- vagrant init; vagrant up
In this example we are converting the HDP Sandbox to be used in this way. But the howto will work with any existing VM.

	# This file contains the minimal code needed to set up a tidymodels flavour for
	# mlflow, along with unit tests. There are three issues that need to be addressed:
	#
	# * we currently use workflows:::predict.workflow, which will NOT be accepted by
	# CRAN. We should ask that the Tidymodels team make this function available in
	# the workflows NAMESPACE.
	# * we need to ensure that the Python CLI can support the new R flavour. There is
	# a commented-out unit test for this.
	# * the unit tests call on packages through `library`, and this could be cleaned up.

	user www-data;
	worker_processes auto;
	pid /run/nginx.pid;

	events {
	worker_connections 768;
	# multi_accept on;
	}

	http {

	#!/bin/bash

	DIR="$@"

	for f in $DIR*.NEF
	do
	name=`echo "$f" \| sed -e "s/.NEF$//g"`
	exiftool -b -JpgFromRaw "$f" > "${name}.jpg"
	done

	#!/bin/bash
	set -x -e

	# AWS EMR bootstrap script
	# for installing open-source R (www.r-project.org) with RHadoop packages and RStudio on AWS EMR
	#
	# tested with AMI 4.0.0 (hadoop 2.6.0)
	#
	# schmidbe@amazon.de
	# 24. September 2014

	Ansible playbook to setup HTTPS using Let's encrypt on nginx.

	The Ansible playbook installs everything needed to serve static files from a nginx server over HTTPS.
	The server pass A rating on [SSL Labs](https://www.ssllabs.com/).

	To use:
	1. Install [Ansible](https://www.ansible.com/)
	2. Setup an Ubuntu 16.04 server accessible over ssh
	3. Create `/etc/ansible/hosts` according to template below and change example.com to your domain
	4. Copy the rest of the files to an empty directory (`playbook.yml` in the root of that folder and the rest in the `templates` subfolder)

	#!/bin/bash

	# These variables can be overwritten using the arguments below
	VERSION="1.1.463"
	# drwho is listed as user in YARN's Resource Manager UI.
	USER="drwho"
	# Depending on where the EMR cluster lives, you might have to change this to avoid security issues.
	# To change the default password (and user), use the arguments bellow.
	# If the cluster is not visible on the Internet, you can just leave the defaults for convenience.
	PASS="tardis"