khttp Documentation Release jkcclemens

Transcription

1 khttp Documentation Release jkcclemens May 12, 2017

2

3 Contents 1 Feature support 3 2 User guide Installation khttp s differences Quickstart Advanced usage i

4 ii

5 khttp is a Mozilla Public License 2.0 licensed library, written in Kotlin, inspired by requests, for human beings. Java is bad at HTTP. Really bad. The tools provided are functionally broken. Unfortunately, by extension, Kotlin is also bad at HTTP. Yet, more and more, the need to interact with APIs and perform basic HTTP requests shows up in programs, but the broken tools in Java make it a big fuss. This isn t how it should be. To combat this, libraries like Unirest have popped up, but they introduce massive amounts of overhead to just make a simple GET request. khttp attempts to solve this problem, by mimicking the Python requests module. khttp uses only the org.json. json library and native Java. There s nothing else. No bullshit. val r = get(" auth=basicauthorization("user", "pass")) r.statuscode // 200 r.headers["content-type"] // "application/json; charset=utf-8" r.text // """{"type": "User"...""" r.jsonobject // org.json.jsonobject See similar code, without khttp. Contents 1

6 2 Contents

7 CHAPTER 1 Feature support khttp is ready for today s web. International domains and URLs Sessions with cookie persistence Basic authentication Elegant key/value cookies Automatic decompression Unicode response bodies Connection timeouts 3

8 4 Chapter 1. Feature support

9 CHAPTER 2 User guide Installation khttp is a new library, and it isn t on Maven Central, yet. Once it matures a bit, it will be looked into. JitPack JitPack is a great service that will build GitHub projects and offer them as a repository for free. If you want to use khttp in a project, use JitPack. Maven Add the JitPack repository <repository> <id>jitpack.io</id> <url> </repository> Add the dependency <dependency> <groupid>com.github.jkcclemens</groupid> <artifactid>khttp</artifactid> <version>-snapshot</version> </dependency> As khttp has no official releases or tags at the moment, JitPack uses -SNAPSHOT to denote the latest commit. 5

10 Get the code khttp is actively developed on GitHub, where the code is always available. You can either clone the public repository: $ git clone git://github.com/jkcclemens/khttp.git Download the tarball: $ curl -OL Or, download the zipball: $ curl -OL khttp s differences While khttp may be made in requests image, it is different in many ways. This part of the documentation explores some important ways in which it is different. It also touches on the differences in how HTTP requests are handled on a lower level. Notable variations from requests khttp attempts to be as close to the requests module as possible, but there are some variations, especially in features that have yet to be implemented. Some parameters are missing The missing parameters from request methods are proxies, verify, and cert. These are planned for implementation. The json() function is split into two properties: jsonobject and jsonarray There is no unifying JSON interface that both JSONObject and JSONArray implement. Rather than creating a wrapper that requires casting and type-checking, there are simply two properties instead of one function. Notable variations from plain ol Java These are variations in how HTTP requests are handled. Obviously the methods in which the requests are made are very, very different. HTTP status codes indicative of errors don t throw exceptions HttpURLConnections really don t like it when there are bad status codes. If you try to get the InputStream of any request that has an error status code, an exception will be thrown, making it so the content can t even be seen. khttp rectifies this. Go ahead, try println(get(" for a pretty picture of a teapot. Of course, if you were using these exceptions to check for errors, now just check the statuscode property. Cookies persist across redirects Java really, really doesn t handle cookies well. Cookies aren t even handled unless a global manager for all requests is set. That s dumb. 6 Chapter 2. User guide

11 khttp allows for the setting and retrieving of cookies, and they persist through redirects, if redirects are allowed. If a POST request is made to a login form, and a cookie is set before being redirected back to the homepage, that cookie will be available in the homepage response. All HTTP methods are allowed, even ones that don t exist Plain ol Java limits requests to the HTTP methods that existed at the time of the creation of the HTTP classes. This means that some HTTP methods like PATCH aren t allowed. khttp overrides this, allowing any method to be used. Quickstart Ready to get started with a better way to make requests? This part of the documentation aims to get you up and running in no time. Let s start with some simple examples, analogous to the ones found in requests quickstart guide. Make a request Making a request with khttp is very simple. Start by importing the request method for the type of request you want to make. For this example, we ll be using GET, so let s import that. import khttp.get Now, let s try to get a webpage. For example, let s get GitHub s public timeline. val r = get(" Now, we have a Response object called r. We can get all the information we need from this object. khttp s API is straightforward, so all forms of HTTP requests are pretty obvious. For example, here s a quick POST request. import khttp.post post(" data = mapof("key" to "value")) Easy. khttp supports every other HTTP request, as well. import khttp.* var r = put(" data = mapof("key" to "value")) r = delete(" r = head(" r = options(" If, for some reason, you need a nonstandard request type, that s supported too. import khttp.request val r = request("nonstandard", " Note: From here on out, the import statements are going to be dropped from the code snippets. They should all be easy enough to figure out Quickstart 7

12 Passing parameters in URLs You often want to send some sort of data in the URL s query string. If you were constructing the URL by hand, this data would be given as key/value pairs in the URL after a question mark, e.g. httpbin.org/get?key=val. khttp allows you to provide these arguments as an object, using the params argument. As an example, if you wanted to pass key1=value1 and key2=value2 to httpbin.org/get, you would use the following code: val payload = mapof("key1" to "value1", "key2" to "value2") val r = get(" params=payload) Note that there is a Parameters class that can be used, as well. There s no real advantage to using it, but it is an option. Note that FormParameters is used for data, not URL parameters (although it will still work as URL parameters if given to the params argument). You can see that the URL has been correctly encoded by printing the URL. Note: At the moment, only Strings can be passed as parameters. In the future, Any will be allowed, which will enable Collections to be passed. println(r.url) // Response content We can read the content of the server s response. Consider the GitHub timeline again: val r = get(" println(r.text) // [{"repository":{"open_issues":0,"url":" When you make a request, khttp makes educated guesses about the encoding of the response based on the HTTP headers. The text encoding guessed by khttp is used when you access r.text. You can find out what encoding khttp is using, and change it, using the r.encoding property: println(r.encoding) // UTF-8 r.encoding = Charsets.ISO_8859_1 If you change the encoding, khttp will use the new value of r.encoding whenever you call r.text. You might want to do this in any situation where you can apply special logic to work out what the encoding of the content will be. For example, HTTP and XML have the ability to specify their encoding in their body. In situations like this, you should use r.raw to find the encoding, and then set r.encoding. This will let you use r.text with the correct encoding. Binary response content You can also access the response body as a ByteArray, for non-text requests: r.content // ByteArray The gzip and deflate transfer-encodings are automatically decoded for you. 8 Chapter 2. User guide

13 JSON response content In case you re dealing with JSON data, khttp will use org.json.json to provide two properties: jsonobject and jsonarray. val r = get(" println(r.jsonarray) // [{"actor":{"avatar_url":" Note that if you attempt to access jsonobject but the content is an array, an exception will be thrown and vice versa. If the content is not JSON, an exception will also be thrown. It should be noted that the success of calls to these properties does not indicate the success of the response. Some servers may return a JSON object in a failed response (e.g. error details with HTTP 500). Such JSON will be decoded and returned. To check that a request is successful, check that r.statuscode is what you expect. Raw response content In the rare case that you d like to get the raw InputStream response from the server, you can access r.raw. If you want to do this, make sure you set stream to true in your initial request. Once you do, you can do this: val r = get(" stream=true) r.raw // InputStream r.raw.read() // 91 In general, however, you should use a pattern like this to save what is being streamed to a file: for (chunk in r.contentiterator(chunksize)) { file.appendbytes(chunk) } When streaming a download, the above is the preferred and recommended way to retrieve the content. Custom headers If you d like to add HTTP headers to a request, simply pass in a Map to the headers parameter. For example, accessing a super secret API: val r = get(" headers=mapof("x-api-key" to "secret")) Some headers may be overwritten depending on context. For example, if the json argument is specified, the Content- Type header will be forced to application/json. khttp does not change behavior based on any specified request headers. It does change behavior based on response headers. More complicated POST requests Typically, you want to send some form-encoded data much like an HTML form. To do this, simply pass a Map to the data argument. Your Map of data will automatically be form-encoded when the request is made: 2.3. Quickstart 9

14 val payload = mapof("key1" to "value1", "key2" to "value2") val r = post(" data=payload) println(r.text) /* {... "form": { "key1": "value1", "key2": "value2" },... } */ There are many times that you want to send data that is not form-encoded. If you pass in any object except for a Map, that data will be posted directly (via the tostring() method). For example, the GitHub API v3 accepts JSON-Encoded POST/PATCH data: val url = " val payload = mapof("some" to "data") val r = post(url, data=jsonobject(payload)) Instead of encoding the JSON yourself, you can also pass it directly using the json parameter, and it will be encoded automatically: val url = " val payload = mapof("some" to "data") r = post(url, json=payload) POST a multipart-encoded file khttp makes it simple to upload multipart-encoded files: val url = " val files = listof(file("report.xls").filelike()) val r = post(url, files = files) r.text /* {... "files": { "report.xls": "<censored...binary...data>" },... } */ You can set the filename explicitly: val url = " val files = listof(file("report.xls").filelike(name = "best_report.xls")) 10 Chapter 2. User guide

15 val r = post(url, files = files) r.text /* {... "files": { "best_report.xls": "<censored...binary...data>" },... } */ If you want, you can send strings to be received as files: val url = " val files = listof("some,data,to,send\nanother,row,to,send\n".filelike(name = "report. csv")) val r = post(url, files = files) r.text /* {... "files": { "report.csv": "some,data,to,send\nanother,row,to,send\n" },... } */ The files parameter is a list of FileLike objects. These objects support all of the methods of uploading files available in requests, but they have a slightly different syntax to be more statically-typed. The filelike(name: String =...) extension function is available on File, Path, and String. This extension function will create FileLike objects in a convenient manner. You can also use the FileLike constructor to create a suitable object. In the event that you are posting a very large file as a multipart/form-data request, you may want to stream the request. By default, khttp does not support this. Response status codes We can check the response status code: val r = get(" r.statuscode // 200 Response headers We can view the server s response headers using a Map: r.headers // {Server=nginx, Access-Control-Allow-Origin=*, Access-Control-Allow- Credentials=true, Connection=keep-alive, Content-Length=235, Date=Wed, 21 Oct :19:06 GMT, Content-Type=application/json} 2.3. Quickstart 11

16 The Map is special, though: it s made just for HTTP headers. According to RFC 7230, HTTP header names are case-insensitive. So, we can access the headers using any capitalization we want: headers["content-type"] // application/json r.headers.get("content-type") // application/json It is also special in that the server could have sent the same header multiple times with different values, but khttp combines them so they can be represented in the Map within a single mapping, as per RFC 7230: A recipient MAY combine multiple header fields with the same field name into one field-name: fieldvalue pair, without changing the semantics of the message, by appending each subsequent field value to the combined field value in order, separated by a comma. Cookies If a response contains some Cookies, you can quickly access them: val url = " val r = get(url) r.cookies["example_cookie_name'] // example_cookie_value Note that khttp will keep cookies persistent throughout redirects. To send your own cookies to the server, you can use the cookies parameter: val url = " val cookies = mapof("cookies_are" to "working") val r = get(url, cookies=cookies) r.text /* { "cookies": { "cookies_are": "working" } } */ Redirection and history By default khttp will perform location redirection for all verbs except for HEAD. If redirect for HEAD requests are desired, you can set the allowredirects parameter in the request to true. We can use the history property of the Response object to track redirection. The history list contains the Response objects that were created in order to complete the request. The list is sorted from the oldest to the most recent response. For example, GitHub redirects all HTTP requests to HTTPS: 12 Chapter 2. User guide

17 val r = get(" r.url // r.statuscode // 200 r.history // [<Response [301]>] If you want to disable redirection handling, you can do so with the allowredirects parameter. val r = get(" allowredirects = false) r.statuscode // 301 r.history // [] Timeouts You can tell khttp to stop waiting for a response after a given number of seconds with the timeout parameter. get(" timeout=0.001) /* java.net.sockettimeoutexception: connect timed out at java.net.plainsocketimpl.socketconnect(native Method) at java.net.abstractplainsocketimpl.doconnect(abstractplainsocketimpl. java:345) at java.net.abstractplainsocketimpl.connecttoaddress(abstractplainsocketimpl. java:206) at java.net.abstractplainsocketimpl.connect(abstractplainsocketimpl.java:188) at java.net.sockssocketimpl.connect(sockssocketimpl.java:392) at java.net.socket.connect(socket.java:589) at sun.net.networkclient.doconnect(networkclient.java:175) at sun.net. at sun.net. at sun.net. at sun.net. at sun.net. at sun.net. getnewhttpclient(httpurlconnection.java:1168) at sun.net. plainconnect0(httpurlconnection.java:1104) at sun.net. java:998) at sun.net. java:932) at khttp.responses.genericresponse.openredirectingconnection(genericresponse. kt:35) at khttp.responses.genericresponse.getconnection(genericresponse.kt:73) at khttp.khttp.request(khttp.kt:53) at khttp.khttp.get(khttp.kt:22) at khttp.khttp.get$default(khttp.kt:21) */ 2.3. Quickstart 13

18 Advanced usage This document covers some of khttp s more advanced features. Streaming uploads khttp supports streaming uploads, which allow you to send large streams or files without reading them into memory. To stream and upload, simply provide a File object for your body: val file = File("massive-body") post(" data = file) Request and response objects Whenever a call is made to khttp.get() and friends you are doing two major things. First, you are constructing a Request object which will be sent off to a server to request or query some resource. Second, a Response object is generated once khttp gets a response back from the server. The Response object contains all of the information returned by the server and also contains the Request object you created originally. Here is a simple request to get some very important information from Wikipedia s servers: val r = khttp.get(" If we want to access the headers the server sent back to us, we do this: r.headers // {X-Cache=cp1053 hit (6), cp1053 frontend hit (229), Server=nginx/1.9.4, X-Content- Type-Options=nosniff, Connection=keep-alive, Last-Modified=Sun, 08 Nov :49:30 GMT, Date=Wed, 11 Nov :53:20 GMT, Via=1.1 varnish, 1.1 varnish, Accept-Ranges=bytes, X-Varnish= , , X-UA- Compatible=IE=Edge, Strict-Transport-Security=max-age= ; includesubdomains; preload, Cache-Control=private, s-maxage=0, max-age=0, must-revalidate, Content- language=en, Content-Encoding=gzip, Vary=Accept-Encoding,Cookie, Content- Length=69400, X-Analytics=page_id=18942;ns=0;WMF-Last-Access=11-Nov-2015;https=1, Age=237769, X-Powered-By=HHVM/3.6.5, Content-Type=text/html; charset=utf-8} However, if we want to get the headers we sent the server, we simply access the request, and then the request s headers: r.request.headers // {Accept=*/*, Accept-Encoding=gzip, deflate, User-Agent=khttp/1.0.0-SNAPSHOT} Asynchronous Requests An asynchronous request can be performed by prefixing the desired HTTP method with the keyword async. The response information is passed through the onresponse callback parameter which provides a reference to a Response object. On the other hand, error information is passed through the onerror callback parameter which provides a reference to a Throwable object. The default onresponse callback simply consumes the Response object while the default onerror callback throws the Throwable object. Usage without specifying an onerror callback is as follows: khttp.async.get(" onresponse = { println("status Code: $statuscode") println("response Text: $text") 14 Chapter 2. User guide

19 }) // OR... khttp.async.get(" { println("status Code: $statuscode") println("response Text: $text") } The onerror callback can be included as follows: khttp.async.get(" onerror = { println("error message: $message") }, onresponse = { println("status Code: $statuscode") println("response Text: $text") }) // OR... khttp.async.get(" onerror = { println("error message: $message") }) { println("status Code: $statuscode") println("response Text: $text") } 2.4. Advanced usage 15