Jonas Finnemann Jensen jonasfj

Query `pubspec.yaml` for all packages

When considering introduction of a new field in pubspec.yaml, or wondering how many packages use a certain feature, it can be useful to query all pubspecs from pub.dev. The following notes have some scripts to make this easy.

File `yaml2json.sh`

Helper script to convert YAML to JSON.

Two simple test files for dart.io.

Run dart download.dart
Run dart extract.dart

Have fun..

Example Configuration for tc-worker with script-engine

Example task: https://tools.taskcluster.net/groups/FxNyHL2oRnWBFzWUdqIkGQ/tasks/FxNyHL2oRnWBFzWUdqIkGQ/details

This works with any clientId/accessToken that has the scope: assume:project:autophone:custom-worker-scopes Notice, that you can take the scopes from that role and make them more specific, so workers have fewer permissions.

	// Copyright 2021 Google LLC.
	// SPDX-License-Identifier: Apache-2.0

	import 'dart:math';

	double entropy(String s) {
	final length = s.length.toDouble();

	// TODO: Explore if this could be done faster without any allocations, since strings are often short.
	final frequencies = <int, int>{};

	// Copyright 2020 Google LLC.
	// SPDX-License-Identifier: Apache-2.0

	import 'dart:async';
	import 'dart:io';

	Future<void> main() async {
	var count = 0;
	ProcessSignal.sigint.watch().forEach((s) {
	count++;

	// Copyright 2019 Google LLC.
	// SPDX-License-Identifier: Apache-2.0

	import 'dart:async' show FutureOr;
	import 'dart:async';
	import 'dart:convert' show utf8, json;
	import 'dart:io' show Directory, Process, IOException, File;
	import 'package:http/http.dart' as http;
	import 'package:retry/retry.dart' show retry;
	import 'package:pool/pool.dart' show Pool;

	// Copyright 2019 Google LLC.
	// SPDX-License-Identifier: Apache-2.0
	//
	// Experiments with dart, grpc and googleapis
	// ==========================================
	//
	// 1. Install `protoc`
	// 2. `pub global activate protoc_plugin`
	// 3. `git clone https://github.com/googleapis/googleapis`
	// 4. `protoc --dart_out=grpc:lib/src/generated -I googleapis/ -I /usr/local/include/ googleapis/google/cloud/vision/v1/image_annotator.proto`

	import requests
	from relengapi.lib.permissions import p

	def scope_satisfied(req, scopes):
	"""
	Returns true, if scopes satisfies req
	"""
	return any((
	s == req or (s.endswith('*') and req.startswith(s[:-1])) for s in scopes
	))


	Decent links:
	* https://github.com/drduh/YubiKey-Guide
	* https://blog.josefsson.org/2014/06/23/offline-gnupg-master-key-and-subkeys-on-yubikey-neo-smartcard/
	* http://stafwag.github.io/blog/blog/2015/06/16/using-yubikey-neo-as-gpg-smartcard-for-ssh-authentication/
	* Or just google for gpg, yubikey, ssh, there is a lot of guides.


	## The GPG setup

	#!/bin/bash

	# This script expects AWS credentials:
	# SIGNIN_AWS_ACCESS_KEY_ID
	# SIGNIN_AWS_SECRET_ACCESS_KEY
	# And optionally the TOTP entry name in your yubikey
	# SIGNIN_AWS_YUBIKEY_OATH_NAME
	# Put these environment variables into your .bashrc.local (or .bashrc, if you
	# don't sync dot-files). In your .bashrc you'll also want:
	# alias signin-aws='eval `signin-aws`'

Jonas Finnemann Jensen jonasfj

Query pubspec.yaml for all packages

File yaml2json.sh

Example Configuration for tc-worker with script-engine

Query `pubspec.yaml` for all packages

File `yaml2json.sh`