Skip to content

Instantly share code, notes, and snippets.

@giuliano108
Last active April 2, 2022 18:50
Show Gist options
  • Save giuliano108/43c0341d9a8e07cb87b42fd0eb0868ef to your computer and use it in GitHub Desktop.
Save giuliano108/43c0341d9a8e07cb87b42fd0eb0868ef to your computer and use it in GitHub Desktop.
Clojure 1.11 `iteration` issue?

<EDIT> Kind folks on the Clojurians Slack helped me understand how to use iteration. Turns out it works absolutely fine. Slack thread is here.

The problem was my understanding of :somef. I was using it like in this Juxt blog post (which is also incorrect) to signal iteration that there are more pages to fetch. :somef, instead, means "does the page we fetched contain any data?".

:kf is what should be used to stop iteration or letting it continue:

If (kf ret) is non-nil it will be passed to the next step call, else iteration will terminate.

The test below can be fixed by removing :somef altogether as in:

(into [] cat (iteration api :initk 0 :kf :tok :vf :ret))                        

Calling :tok on the last page returns nil, because the map doesn't contain :tok at all. In that case iteration stops.

</EDIT>

iteration doesn't return the last "page" of data. This seems to be by design, the docs say:

Iff (somef ret) is true, (vf ret) will be included in the iteration, else iteration will terminate and vf/kf will not be called.

The test below is missing items number 6 and 7. The API returns them in the last page, setting :more to false (meaning that pagination ended and there's no more data to fetch).

(ns user
(:require [clojure.test :refer :all]))
(deftest test-iteration
;; paginated API
(let [api (fn [tok]
(condp = tok
0 {:ret [0 1 2] :tok 3 :more true}
3 {:ret [3 4 5] :tok 6 :more true}
6 {:ret [6 7] :more false}
nil))]
(is (= [0 1 2 3 4 5 6 7]
(into [] cat (iteration api :initk 0 :kf :tok :vf :ret :somef :more))))))
; user> (run-tests)
;
; Testing user
;
; FAIL in (test-iteration) (it.clj:14)
; expected: (= [0 1 2 3 4 5 6 7] (into [] cat (iteration api :initk 0 :kf :tok :vf :ret :somef :more)))
; actual: (not (= [0 1 2 3 4 5 6 7] [0 1 2 3 4 5]))
;
; Ran 1 tests containing 1 assertions.
; 1 failures, 0 errors.
; {:test 1, :pass 0, :fail 1, :error 0, :type :summary}
; user>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment