Skip to content

Instantly share code, notes, and snippets.

View drorata's full-sized avatar

Dror Atariah drorata

View GitHub Profile
@drorata
drorata / notebook.ipynb
Created May 10, 2023 09:42
Spark comparing columns
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@drorata
drorata / python-build.20230404093016.30887.log
Last active April 4, 2023 06:35
Fail to build 3.8.9 using Pyenv
/var/folders/y6/skpcb0d11fb0yknzv6934h480000gn/T/python-build.20230404093016.30887 ~/tmp
++(/opt/homebrew/Cellar/pyenv/2.3.16/plugins/python-build/bin/python-build:334): http(): local method=get
++(/opt/homebrew/Cellar/pyenv/2.3.16/plugins/python-build/bin/python-build:335): http(): '[' -n https://www.python.org/ftp/python/3.8.9/Python-3.8.9.tar.xz ']'
++(/opt/homebrew/Cellar/pyenv/2.3.16/plugins/python-build/bin/python-build:336): http(): shift 1
+++(/opt/homebrew/Cellar/pyenv/2.3.16/plugins/python-build/bin/python-build:338): http(): detect_http_client
+++(/opt/homebrew/Cellar/pyenv/2.3.16/plugins/python-build/bin/python-build:345): detect_http_clienlocal client
+++(/opt/homebrew/Cellar/pyenv/2.3.16/plugins/python-build/bin/python-build:346): detect_http_clienfor client in aria2c curl wget
+++(/opt/homebrew/Cellar/pyenv/2.3.16/plugins/python-build/bin/python-build:347): detect_http_clientype aria2c
+++(/opt/homebrew/Cellar/pyenv/2.3.16/plugins/python-build/bin/python-build:346): detect_http_clienfor client
Visual Studio Code (1.74.3, undefined, desktop)
Jupyter Extension Version: 2023.1.1003441034.
Python Extension Version: 2022.20.2.
Workspace folder /Users/username/My Drive/Private/Football analysis
warn 10:36:00.299: Exception while attempting zmq : No native build was found for platform=darwin arch=arm64 runtime=electron abi=106 uv=1 armv=8 libc=glibc node=16.14.2 electron=19.1.8
loaded from: /Users/username/.vscode/extensions/ms-toolsai.jupyter-2023.1.1003441034/out/node_modules/zeromq
User belongs to experiment group 'jupyterTest'
User belongs to experiment group 'jupyterEnhancedDataViewer'
debug 10:36:00.454: Updating cache with Local kernels
@drorata
drorata / gist:608739e1aa4e5f31a62cbd63455c99fd
Created January 9, 2023 06:31
Detailed logs of Jupyter inside VScode
Visual Studio Code (1.74.2, undefined, desktop)
Jupyter Extension Version: 2023.1.1003441034.
Python Extension Version: 2022.20.1.
Workspace folder /Users/username/My Drive/Private/Football analysis
warn 07:29:47.904: Exception while attempting zmq : No native build was found for platform=darwin arch=arm64 runtime=electron abi=106 uv=1 armv=8 libc=glibc node=16.14.2 electron=19.1.8
loaded from: /Users/username/.vscode/extensions/ms-toolsai.jupyter-2023.1.1003441034/out/node_modules/zeromq
User belongs to experiment group 'jupyterTest'
User belongs to experiment group 'jupyterEnhancedDataViewer'
debug 07:29:48.056: Updating cache with Local kernels
Experiment 'pythonaacf' is active
Experiment 'pythonSurveyNotificationcf' is active
> poetry env list --full-path
cwd: ~/My Drive/Private/Football analysis
[DEBUG 2023-0-6 11:45:15.870]: Cached data exists KeyPrefix=Cache_Method_Output_m.addKeyToStorage-Args="global"-Arg-Separator-"WORKSPACE_FOLDER_INTERPRETER_PATH_/Users/username/My Drive/Private/Football analysis"-Arg-Separator-undefined
[DEBUG 2023-0-6 11:45:15.870]: Cached data exists KeyPrefix=Cache_Method_Output_m.addKeyToStorage-Args="workspace"-Arg-Separator-"autoSelectedWorkspacePythonInterpreter-/Users/username/My Drive/Private/Football analysis"-Arg-Separator-undefined
[DEBUG 2023-0-6 11:45:15.870]: Cached data exists KeyPrefix=Cache_Method_Output_m.addKeyToStorage-Args="workspace"-Arg-Separator-"autoSelectedWorkspacePythonInterpreter-/Users/username/My Drive/Private/Football analysis"-Arg-Separator-undefined
[DEBUG 2023-0-6 11:45:15.871]: Cached data exists KeyPrefix=Cache_Method_Output_m.addKeyToStorage-Args="global"-Arg-Separator-"WORKSPACE_FOLD
@drorata
drorata / description.md
Created December 27, 2022 17:24
MWE - rendering Markdown using a LaTeX template

I am using the function render_latex:

def render_latex(text: str, minimal: bool = True) -> str:
    logger.debug(f"Rendering LaTeX - {text}")
    with LaTeXRenderer() as rendered:
        ast = Document(text)
        if len(ast.children) != 1:
            raise ValueError("Number of children of provided text is not 1")
 for child in ast.children[0].children:
@drorata
drorata / example.ipynb
Created November 11, 2022 07:27
PySpark and time zones
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@drorata
drorata / error.txt
Created November 4, 2022 07:33
Error when trying to install mypy using poetry (from home using wifi+vpn)
Stack trace:
7 ~\.poetry\lib\site-packages\urllib3\response.py:443 in _error_catcher
441│ try:
442│ try:
→ 443│ yield
444│
445│ except SocketTimeout:
6 ~\.poetry\lib\site-packages\urllib3\response.py:566 in read
@drorata
drorata / pyproject.toml
Created October 6, 2022 11:43
pyspark_write_demo
[tool.poetry]
name = "pyspark_write_demo"
version = "0.1.0"
description = ""
authors = ["Dr. Dror"]
[tool.poetry.dependencies]
python = "^3.9"
pyspark = "^3.3.0"
pandas = "^1.5.0"
@drorata
drorata / foo.py
Created September 16, 2022 10:49
Minimal example using abstract class
from abc import ABC, abstractmethod
class EcgDataBase(ABC):
@abstractmethod
def __init__(self):
pass
@abstractmethod
def __iter__(self):