Skip to content

Instantly share code, notes, and snippets.

@ctlllll
ctlllll / longest_chinese_tokens_gpt4o.py
Created May 13, 2024 19:53
Longest Chinese tokens in gpt4o
import tiktoken
import langdetect
T = tiktoken.get_encoding("o200k_base")
length_dict = {}
for i in range(T.n_vocab):
try:
length_dict[i] = len(T.decode([i]))
except:
@augbog
augbog / Free O'Reilly Books.md
Last active May 14, 2024 10:27
Free O'Reilly Books

Free O'Reilly books and convenient script to just download them.

Thanks /u/FallenAege/ and /u/ShPavel/ from this Reddit post

How to use:

  1. Take the download.sh file and put it into a directory where you want the files to be saved.
  2. cd into the directory and make sure that it has executable permissions (chmod +x download.sh should do it)
  3. Run ./download.sh and wee there it goes. Also if you do not want all the files, just simply comment the ones you do not want.
@guweigang
guweigang / git_toturial
Last active May 14, 2024 10:19
git命令大全
git init # 初始化本地git仓库(创建新仓库)
git config --global user.name "xxx" # 配置用户名
git config --global user.email "xxx@xxx.com" # 配置邮件
git config --global color.ui true # git status等命令自动着色
git config --global color.status auto
git config --global color.diff auto
git config --global color.branch auto
git config --global color.interactive auto
git config --global --unset http.proxy # remove proxy configuration on git
git clone git+ssh://git@192.168.53.168/VT.git # clone远程仓库
@avshatalov48
avshatalov48 / Git.md
Last active May 14, 2024 10:12
Основные "повседневные" команды GIT

Основные "повседневные" команды GIT

  • git status
  • git add .
  • git commit -m "GeekBrains > Yii2 > Lesson 4 > Complete"
  • git push
  • git pull
  • git log
  • git config --list
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@padeoe
padeoe / README_hfd.md
Last active May 14, 2024 09:53
CLI-Tool for download Huggingface models and datasets with aria2/wget+git

🤗Huggingface Model Downloader

Considering the lack of multi-threaded download support in the official huggingface-cli, and the inadequate error handling in hf_transfer, this command-line tool smartly utilizes wget or aria2 for LFS files and git clone for the rest.

Features

  • ⏯️ Resume from breakpoint: You can re-run it or Ctrl+C anytime.
  • 🚀 Multi-threaded Download: Utilize multiple threads to speed up the download process.
  • 🚫 File Exclusion: Use --exclude or --include to skip or specify files, save time for models with duplicate formats (e.g., *.bin or *.safetensors).
  • 🔐 Auth Support: For gated models that require Huggingface login, use --hf_username and --hf_token to authenticate.
  • 🪞 Mirror Site Support: Set up with HF_ENDPOINT environment variable.
@ulrikdamm
ulrikdamm / EditPrefab.cs
Last active May 14, 2024 09:51
Unity editor script for better editing of prefabs. Put in Assets/Editor.
using UnityEngine;
using UnityEditor;
using UnityEditor.SceneManagement;
// Adds a "Edit Prefab" option in the Assets menu (or right clicking an asset in the project browser).
// This opens an empty scene with your prefab where you can edit it.
// Put this script in your project as Assets/Editor/EditPrefab.cs
public class EditPrefab {
static Object getPrefab(Object selection) {
@hoangsonww
hoangsonww / Blockchain.py
Created April 8, 2024 04:05
A basic blockchain structure in Python with proof-of-work consensus, transaction handling, and wallet functionalities.
import hashlib
import json
from time import time
import binascii
from collections import OrderedDict
from uuid import uuid4
from cryptography.hazmat.primitives import hashes
from cryptography.hazmat.primitives.asymmetric import ec
from cryptography.hazmat.primitives import serialization
from cryptography.exceptions import InvalidSignature
@minikomi
minikomi / index.html
Last active May 14, 2024 09:38
Japan geojson - albers
<!DOCTYPE html>
<meta charset="utf-8">
<body>
<script src="http://d3js.org/d3.v2.min.js?2.10.0"></script>
<script>
var width = 960,
height = 700;
var svg = d3.select("body").append("svg")
@mweibel
mweibel / passport-mock.js
Last active May 14, 2024 09:32
Mock passport.js with an own strategy
/**
* Author: Michael Weibel <michael.weibel@gmail.com>
* License: MIT
*/
var passport = require('passport')
, StrategyMock = require('./strategy-mock');
module.exports = function(app, options) {
// create your verify function on your own -- should do similar things as