tom white dribnet

A criticism of "On the Dangers of Stochastic Parrots: Can Languae Models be Too Big"

Yoav Goldberg, Jan 23, 2021.

The FAccT paper "On the Dangers of Stochastic Parrots: Can Languae Models be Too Big" by Bender, Gebru, McMillan-Major and Shmitchell has been the center of a controversary recently. The final version is now out, and, owing a lot to this controversary, would undoubtly become very widely read. I read an earlier draft of the paper, and I think that the new and updated final version is much improved in many ways: kudos for the authors for this upgrade. I also agree with and endorse most of the content. This is important stuff, you should read it.

However, I do find some aspects of the paper (and the resulting discourse around it and around technology) to be problematic. These weren't clear to me when initially reading the first draft several months ago, but they became very clear to me now. These points are for the most part

	from diffusers import DiffusionPipeline, DPMSolverMultistepScheduler
	import torch


	#////////////////////////////////////////////////////////////////
	guidance_scale=7.5
	steps_base=50
	steps_refiner=50
	width=1024
	height=1024

	"""
	stable diffusion dreaming
	creates hypnotic moving videos by smoothly walking randomly through the sample space

	example way to run this script:

	$ python stablediffusionwalk.py --prompt "blueberry spaghetti" --name blueberry

	to stitch together the images, e.g.:
	$ ffmpeg -r 10 -f image2 -s 512x512 -i blueberry/frame%06d.jpg -vcodec libx264 -crf 10 -pix_fmt yuv420p blueberry.mp4


	#Evolution Strategies with Keras
	#Based off of: https://blog.openai.com/evolution-strategies/
	#Implementation by: Nicholas Samoray

	#README
	#Meant to be run on a single machine
	#APPLY_BIAS is currently not working, keep to False
	#Solves Cartpole as-is in about 50 episodes
	#Solves BipedalWalker-v2 in about 1000

	<!DOCTYPE html>
	<html>

	<head>
	<script data-require="jquery@3.1.1" data-semver="3.1.1" src="https://ajax.googleapis.com/ajax/libs/jquery/3.1.1/jquery.min.js"></script>
	<script data-require="tether@*" data-semver="1.4.0" src="https://cdnjs.cloudflare.com/ajax/libs/tether/1.4.0/js/tether.min.js"></script>
	<link data-require="bootstrap@4.0.5" data-semver="4.0.5" rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/4.0.0-alpha.6/css/bootstrap.min.css" />
	<script data-require="bootstrap@4.0.5" data-semver="4.0.5" src="https://maxcdn.bootstrapcdn.com/bootstrap/4.0.0-alpha.5/js/bootstrap.min.js"></script>
	<script data-require="react@*" data-semver="15.5.0" src="https://cdnjs.cloudflare.com/ajax/libs/react/15.5.0/react.min.js"></script>
	<script data-require="react@*" data-semver="15.5.0" src="https://cdnjs.cloudflare.com/ajax/libs/react/15.5.0/react-dom.min.js"></script>

	import mxnet as mx
	import numpy as np
	import cPickle
	import cv2

	def extractImagesAndLabels(path, file):
	f = open(path+file, 'rb')
	dict = cPickle.load(f)
	images = dict['data']
	images = np.reshape(images, (10000, 3, 32, 32))

	'''This script goes along the blog post
	"Building powerful image classification models using very little data"
	from blog.keras.io.
	It uses data that can be downloaded at:
	https://www.kaggle.com/c/dogs-vs-cats/data
	In our setup, we:
	- created a data/ folder
	- created train/ and validation/ subfolders inside data/
	- created cats/ and dogs/ subfolders inside train/ and validation/
	- put the cat pictures index 0-999 in data/train/cats

	import numpy as np
	import gym
	from gym.spaces import Discrete, Box
	from gym.wrappers import Monitor
	from keras.models import Sequential
	from keras.layers import Dense, Activation, Flatten


	# ================================================================
	# Policies