pdurbin · January 12, 2016 19:40
diff --git a/README.md b/README.md
diff --git a/create-ss.png b/create-ss.png
diff --git a/dataverse-logo.png b/dataverse-logo.png
diff --git a/dataversesit.png b/dataversesit.png
diff --git a/good-bad.png b/good-bad.png
diff --git a/hippo-oath.jpg b/hippo-oath.jpg
diff --git a/index.html b/index.html
 <!DOCTYPE html>
 <html>
  <head>
    <title>2016-01-12 Dataverse API Testing</title>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
    <style type="text/css">
      @import url(http://fonts.googleapis.com/css?family=Yanone+Kaffeesatz);
      @import url(http://fonts.googleapis.com/css?family=Droid+Serif:400,700,400italic);
      @import url(http://fonts.googleapis.com/css?family=Ubuntu+Mono:400,700,400italic);

      body { font-family: 'Droid Serif'; }
      h1, h2, h3 {
        font-family: 'Yanone Kaffeesatz';
        font-weight: normal;
      }
      .remark-code, .remark-inline-code { font-family: 'Ubuntu Mono'; }
      .remark-slide-content { font-size: 25px; }
      .title h1 { font-size: 70px; }
      .agenda { font-size: 26px; }
      .footnote {
        font-size: 15px;
        position: absolute;
        bottom: 1em;
        left: 3em;
        right: 1em;
      }
      .quote { font-size: 18px; }
    </style>
  </head>
  <body>
    <textarea id="source">

 class: center, middle, title

 # API Testing with REST Assured

 <img src="dataverse-logo.png" width="300px"/></br>
 <img src="rest-assured-logo.png" width="300px"/><br/>

 </br>

 .left[
 <img src="philipdurbin.jpg" width="60" align="left" style="border:5px solid transparent" />
 Philip Durbin ([@philipdurbin](https://twitter.com/philipdurbin))  
 http://www.iq.harvard.edu/people/philip-durbin  
 ]

 .right[<img src="iqss-logo.png" width="100" align="left" style="border:5px solid transparent" />]

 2016-01-12

 ---

 # Agenda

 - How to write and run API tests from Netbeans.
 - How to execute tests on a server such as http://phoenix.dataverse.org
 - Tips and tricks on using REST Assured.
 - Overview of automated testing ideas: https://github.com/IQSS/dataverse/issues/2746

 ---


 # What is an API?

 - Application Programming Interface.
 - Technically, APIs are everywhere. We're talking about testing REST (HTTP) APIs.

 <img src="ojs.png" width="700px"/>

 ---

 # The good and bad of testing APIs

 - Good
    - Real world ("acceptance tests").
      - Can you create a user?
      - Can you upload a file?
    - Much easier than testing the UI.
    - Is business logic centralized? Any GUI-only features?
 - Bad
    - Database, etc. must be running.
        - Where are these tests going to run?
    - Many steps required to test certain scenarios (i.e. deleting a file).
    - Hard to measure coverage, unlike unit tests: https://coveralls.io/github/IQSS/dataverse

 ---

 # How do you test an API?

 - One off tests
    - Hit it with `curl` a few times in development. Hit the "up" arrow as needed.
 - Automate the tests
    - Use REST Assured or your favorite testing library.
    - How often do you run them? Where? On your laptop?
 - Run the automated tests on a real server
    - Use Jenkins or similar to trigger the tests after a build.

 ---

 # Is there an API Test already?

 <img src="navigate.png" width="600px"/>

 ---

 # DataversesIT ("integration test")

 <img src="dataversesit.png" width="600px"/>

 src/test/java/edu/harvard/iq/dataverse/api

 ---

 # Creating an Integration Test Step 1

 <img src="nav-to-ss.png" width="600px"/>

 ---

 # Creating an Integration Test Step 2

 <img src="create-ss.png" width="400px" align="right"/>

 - FooTest vs. FooIT
  - IT checkbox
 - JUnit
 - Code generation

 ---

 # Side Effects of API Testing

 <img src="tests-ran-fine.jpg" width="350px"/> <img src="hippo-oath.jpg" width="350px"/>

 - Do no harm.
 - Clean up after yourself.

 ---

 # Running an integration test

 <img src="run-test.png" width="700px"/>

    mvn test -Dtest=DataversesIT

 ---

 # What is the test about?

 ### `testAttemptToCreateDuplicateAlias`

 <img src="stay-saved.jpg" width="250px" align="right"/> 

 - Reads a bit like a user story.
    - "Alice attempts to create a dataverse with an alias that's the upper case version of an existing alias. This should fail and report the error back to Alice."

 <img src="only-one.png" width="150px"/> 

 cfa vs. CFA vs. CfA  
 https://github.com/IQSS/dataverse/issues/2598

 ---

 # Is your API test doing too much?

 #### `testCreateDatasetUploadFileDownloadFileEditTitle`

 <img src="perl6.gif" width="300px" align="right"/> 

 Assert that:

 - dataverse created
 - dataset created via SWORD
 - datasets listed via SWORD
 - file uploaded
 - download file by id
 - delete file
 - update title
 - etc.

 src/test/java/edu/harvard/iq/dataverse/api/SwordIT.java

 ---

 # JUnit Assertions

 .center[
 <img src="good-bad.png" width="400px"/> 
 ]

 - `assertEquals(200, attemptToDoSomethingGood)`
 - `assertEquals(400, attemptToDoSomethingBad)`

 ---

 # REST Assured

 "Java DSL for easy testing of REST services" -- https://github.com/jayway/rest-assured

 <img src="rest-assured-logo.png" width="200px"/>

 "Jayway is a software innovation consultancy" -- http://jayway.com

 <img src="jayway-logo.png" width="200px"/>

 - status codes
 - JSON response
 - XML response

 ---

 # REST Assured Assertions

    Response listDatasetsResponse = UtilIT.listDatasetsViaSword(dataverseAlias1, apiToken1);
    listDatasetsResponse.prettyPrint();
    listDatasetsResponse.then().assertThat()
            .statusCode(OK.getStatusCode())
            .body("feed.dataverseHasBeenReleased", equalTo("false"))
            .body("feed.entry[0].title", equalTo(initialDatasetTitle));

 Assert that:

 - Status code is OK (200).
 - "dataverseHasBeenReleased" is false
 - title matches expected string

 Return type: `com.jayway.restassured.response.Response`

 src/test/java/edu/harvard/iq/dataverse/api/SwordIT.java

 ---


 # Testing on a server

 <img src="trend.png" width="700px"/>

 https://build.hmdc.harvard.edu:8443

 ---

 # Triggering tests from Jenkins

 - Build
    - Build a war file and package setup scripts.
    - https://build.hmdc.harvard.edu:8443/job/phoenix.dataverse.org-build-4.2.3/
 - Deploy
    - Drop database, deploy war file, setup, curl tests.
    - https://build.hmdc.harvard.edu:8443/job/phoenix.dataverse.org-deploy/
 - Test
    - REST Assured tests
    - https://build.hmdc.harvard.edu:8443/job/phoenix.dataverse.org-apitest-4.2.3/

 ---

 # Only a piece of the testing puzzle

 Improve automated testing - https://github.com/IQSS/dataverse/issues/2746

 <img src="puzzle.jpg" width="350px" align="right"/>

 - API Tests
 - Unit Tests 
 - [The Testable Command](https://github.com/IQSS/dataverse/blob/1d7eadcb89ccebb4990c62757c84ec705f13853d/doc/theTestableCommand/TheTestableCommand.md)
 - Selenium (browser)
 - Performance
 - Security
 - etc.

 .footnote[
 https://www.flickr.com/photos/horiavarlan/4273913228
 ]

    </textarea>
    <script src="http://gnab.github.io/remark/downloads/remark-latest.min.js" type="text/javascript">
    </script>
    <script type="text/javascript">
      var slideshow = remark.create();
    </script>
  </body>
 </html>
diff --git a/iqss-logo.png b/iqss-logo.png
diff --git a/jayway-logo.png b/jayway-logo.png
diff --git a/nav-to-ss.png b/nav-to-ss.png
diff --git a/navigate.png b/navigate.png
diff --git a/ojs.png b/ojs.png
diff --git a/only-one.png b/only-one.png
diff --git a/perl6.gif b/perl6.gif
diff --git a/philipdurbin.jpg b/philipdurbin.jpg
diff --git a/puzzle.jpg b/puzzle.jpg
diff --git a/rest-assured-logo.png b/rest-assured-logo.png
diff --git a/run-test.png b/run-test.png
diff --git a/stay-saved.jpg b/stay-saved.jpg
diff --git a/tests-ran-fine.jpg b/tests-ran-fine.jpg
diff --git a/trend.png b/trend.png
	<!DOCTYPE html>
	<html>
	<head>
	<title>2016-01-12 Dataverse API Testing</title>
	<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
	<style type="text/css">
	@import url(http://fonts.googleapis.com/css?family=Yanone+Kaffeesatz);
	@import url(http://fonts.googleapis.com/css?family=Droid+Serif:400,700,400italic);
	@import url(http://fonts.googleapis.com/css?family=Ubuntu+Mono:400,700,400italic);

	body { font-family: 'Droid Serif'; }
	h1, h2, h3 {
	font-family: 'Yanone Kaffeesatz';
	font-weight: normal;
	}
	.remark-code, .remark-inline-code { font-family: 'Ubuntu Mono'; }
	.remark-slide-content { font-size: 25px; }
	.title h1 { font-size: 70px; }
	.agenda { font-size: 26px; }
	.footnote {
	font-size: 15px;
	position: absolute;
	bottom: 1em;
	left: 3em;
	right: 1em;
	}
	.quote { font-size: 18px; }
	</style>
	</head>
	<body>
	<textarea id="source">

	class: center, middle, title

	# API Testing with REST Assured

	<img src="dataverse-logo.png" width="300px"/></br>
	<img src="rest-assured-logo.png" width="300px"/><br/>

	</br>

	.left[
	<img src="philipdurbin.jpg" width="60" align="left" style="border:5px solid transparent" />
	Philip Durbin ([@philipdurbin](https://twitter.com/philipdurbin))
	http://www.iq.harvard.edu/people/philip-durbin
	]

	.right[<img src="iqss-logo.png" width="100" align="left" style="border:5px solid transparent" />]

	2016-01-12

	---

	# Agenda

	- How to write and run API tests from Netbeans.
	- How to execute tests on a server such as http://phoenix.dataverse.org
	- Tips and tricks on using REST Assured.
	- Overview of automated testing ideas: https://github.com/IQSS/dataverse/issues/2746

	---


	# What is an API?

	- Application Programming Interface.
	- Technically, APIs are everywhere. We're talking about testing REST (HTTP) APIs.

	<img src="ojs.png" width="700px"/>

	---

	# The good and bad of testing APIs

	- Good
	- Real world ("acceptance tests").
	- Can you create a user?
	- Can you upload a file?
	- Much easier than testing the UI.
	- Is business logic centralized? Any GUI-only features?
	- Bad
	- Database, etc. must be running.
	- Where are these tests going to run?
	- Many steps required to test certain scenarios (i.e. deleting a file).
	- Hard to measure coverage, unlike unit tests: https://coveralls.io/github/IQSS/dataverse

	---

	# How do you test an API?

	- One off tests
	- Hit it with `curl` a few times in development. Hit the "up" arrow as needed.
	- Automate the tests
	- Use REST Assured or your favorite testing library.
	- How often do you run them? Where? On your laptop?
	- Run the automated tests on a real server
	- Use Jenkins or similar to trigger the tests after a build.

	---

	# Is there an API Test already?

	<img src="navigate.png" width="600px"/>

	---

	# DataversesIT ("integration test")

	<img src="dataversesit.png" width="600px"/>

	src/test/java/edu/harvard/iq/dataverse/api

	---

	# Creating an Integration Test Step 1

	<img src="nav-to-ss.png" width="600px"/>

	---

	# Creating an Integration Test Step 2

	<img src="create-ss.png" width="400px" align="right"/>

	- FooTest vs. FooIT
	- IT checkbox
	- JUnit
	- Code generation

	---

	# Side Effects of API Testing

	<img src="tests-ran-fine.jpg" width="350px"/> <img src="hippo-oath.jpg" width="350px"/>

	- Do no harm.
	- Clean up after yourself.

	---

	# Running an integration test

	<img src="run-test.png" width="700px"/>

	mvn test -Dtest=DataversesIT

	---

	# What is the test about?

	### `testAttemptToCreateDuplicateAlias`

	<img src="stay-saved.jpg" width="250px" align="right"/>

	- Reads a bit like a user story.
	- "Alice attempts to create a dataverse with an alias that's the upper case version of an existing alias. This should fail and report the error back to Alice."

	<img src="only-one.png" width="150px"/>

	cfa vs. CFA vs. CfA
	https://github.com/IQSS/dataverse/issues/2598

	---

	# Is your API test doing too much?

	#### `testCreateDatasetUploadFileDownloadFileEditTitle`

	<img src="perl6.gif" width="300px" align="right"/>

	Assert that:

	- dataverse created
	- dataset created via SWORD
	- datasets listed via SWORD
	- file uploaded
	- download file by id
	- delete file
	- update title
	- etc.

	src/test/java/edu/harvard/iq/dataverse/api/SwordIT.java

	---

	# JUnit Assertions

	.center[
	<img src="good-bad.png" width="400px"/>
	]

	- `assertEquals(200, attemptToDoSomethingGood)`
	- `assertEquals(400, attemptToDoSomethingBad)`

	---

	# REST Assured

	"Java DSL for easy testing of REST services" -- https://github.com/jayway/rest-assured

	<img src="rest-assured-logo.png" width="200px"/>

	"Jayway is a software innovation consultancy" -- http://jayway.com

	<img src="jayway-logo.png" width="200px"/>

	- status codes
	- JSON response
	- XML response

	---

	# REST Assured Assertions

	Response listDatasetsResponse = UtilIT.listDatasetsViaSword(dataverseAlias1, apiToken1);
	listDatasetsResponse.prettyPrint();
	listDatasetsResponse.then().assertThat()
	.statusCode(OK.getStatusCode())
	.body("feed.dataverseHasBeenReleased", equalTo("false"))
	.body("feed.entry[0].title", equalTo(initialDatasetTitle));

	Assert that:

	- Status code is OK (200).
	- "dataverseHasBeenReleased" is false
	- title matches expected string

	Return type: `com.jayway.restassured.response.Response`

	src/test/java/edu/harvard/iq/dataverse/api/SwordIT.java

	---


	# Testing on a server

	<img src="trend.png" width="700px"/>

	https://build.hmdc.harvard.edu:8443

	---

	# Triggering tests from Jenkins

	- Build
	- Build a war file and package setup scripts.
	- https://build.hmdc.harvard.edu:8443/job/phoenix.dataverse.org-build-4.2.3/
	- Deploy
	- Drop database, deploy war file, setup, curl tests.
	- https://build.hmdc.harvard.edu:8443/job/phoenix.dataverse.org-deploy/
	- Test
	- REST Assured tests
	- https://build.hmdc.harvard.edu:8443/job/phoenix.dataverse.org-apitest-4.2.3/

	---

	# Only a piece of the testing puzzle

	Improve automated testing - https://github.com/IQSS/dataverse/issues/2746

	<img src="puzzle.jpg" width="350px" align="right"/>

	- API Tests
	- Unit Tests
	- [The Testable Command](https://github.com/IQSS/dataverse/blob/1d7eadcb89ccebb4990c62757c84ec705f13853d/doc/theTestableCommand/TheTestableCommand.md)
	- Selenium (browser)
	- Performance
	- Security
	- etc.

	.footnote[
	https://www.flickr.com/photos/horiavarlan/4273913228
	]

	</textarea>
	<script src="http://gnab.github.io/remark/downloads/remark-latest.min.js" type="text/javascript">
	</script>
	<script type="text/javascript">
	var slideshow = remark.create();
	</script>
	</body>
	</html>