Skip to content

Instantly share code, notes, and snippets.

View anarazel's full-sized avatar

Andres Freund anarazel

View GitHub Profile
@anarazel
anarazel / latency.md
Last active May 25, 2024 21:14
SSD durable write latency

Collecting latency information for durable QD 1 writes, measured using

fio --directory /mnt/t2/fio/ \
  --runtime 3 --time_based \
  --output-format json \
  --overwrite 1 --size=8MB --buffered 0 --bs=4096 --rw=write \
  --name write-dsync --wait_for_previous --sync=dsync \
  --name write-fdatasync --wait_for_previous --fdatasync=1 \
  --name write-nondurable --wait_for_previous \
 | jq '.jobs[] | [.jobname, .write.iops]'
96 isolation/lock-committed-update/control connection
100 isolation/partial-index/control connection
105 isolation/referential-integrity/control connection
120 isolation/detach-partition-concurrently-4/control connection
120 isolation/predicate-gist/control connection
156 isolation/eval-plan-qual-trigger/control connection
160 isolation/project-manager/control connection
160 isolation/temporal-range-integrity/control connection
169 isolation/merge-update/control connection
180 isolation/two-ids/control connection
119: + descr => 'contents of a WAL sumamry file',
sumamry ==> summary
471: + * chunk. If that value is less than MAX_ENTRIES_PER_CHUNK, the corresonding
corresonding ==> corresponding
554: + * combination that is curently being read, and consumed_chunks is the number
curently ==> currently
938: + * 'error_callback_arg' is an opaque arguent to be passed to error_callback.
arguent ==> argument
1293: + * contain the limit_block. We must handle this differenly depending on
differenly ==> differently
import matplotlib
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import sys
def quantplot(df, percentile_limit=0.001, n=10, semilog = False):
quantiles = np.linspace(0+percentile_limit, 1-percentile_limit, num=n*2+1)
alpha = 1/n
bits: 32, method: NON_ATOMIC_METHOD_RACY, concurrency: 1:
throughput per thread: 603.36M/s, total: 603.36M/s
bits: 32, method: NON_ATOMIC_METHOD_RACY, concurrency: 40:
final counters after killing are: 19991546, per-thread counters sum 678064093, diff -658072547 (allowed <= -40), retries 0
throughput per thread: 5.65M/s, total: 226.02M/s
bits: 32, method: NON_ATOMIC_METHOD_RACY, concurrency: 160:
final counters after killing are: 20702649, per-thread counters sum 696222934, diff -675520285 (allowed <= -160), retries 0
throughput per thread: 1.45M/s, total: 232.07M/s
#define _GNU_SOURCE
#include <pthread.h>
#include <signal.h>
#include <stdbool.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <pthread.h>
#include <stdio.h>
#include <stdint.h>
#include <unistd.h>
#include <signal.h>
#include <stdlib.h>
#include <string.h>
#include <stdbool.h>
#ifndef SECS
Jun 13 16:39:39 vm-debian-10 kernel: Linux version 6.4.0-rc6-00029-gec5c40aa4d32 (andres@awork3) (gcc (Debian 12.3.0-1) 12.3.0, GNU ld (GNU Binutils for Debian) 2.40.50.20230611) #124 SMP PREEMPT_DYNAMIC Tue Jun 13 16:12:18 PDT 2023
Jun 13 16:39:39 vm-debian-10 kernel: Command line: root=/dev/vda2 ro systemd.unified_cgroup_hierarchy=1
Jun 13 16:39:39 vm-debian-10 kernel: x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'
Jun 13 16:39:39 vm-debian-10 kernel: x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
Jun 13 16:39:39 vm-debian-10 kernel: x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'
Jun 13 16:39:39 vm-debian-10 kernel: x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'
Jun 13 16:39:39 vm-debian-10 kernel: x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'
Jun 13 16:39:39 vm-debian-10 kernel: x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'
Jun 13 16:39:39 vm-debian-10 kernel: x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User reg
WITH
constants AS (
SELECT
current_setting('server_version') AS version,
current_setting('block_size')::int AS block_size,
current_setting('autovacuum_vacuum_cost_delay') AS cost_delay,
1000 / extract(milliseconds FROM current_setting('autovacuum_vacuum_cost_delay')::interval) AS delays_per_sec,
COALESCE(NULLIF(current_setting('autovacuum_vacuum_cost_limit')::int, -1), current_setting('vacuum_cost_limit')::int) AS cost_limit,
current_setting('vacuum_cost_page_hit')::int AS cost_hit,
current_setting('vacuum_cost_page_miss')::int AS cost_miss,