Load a Document from a URL

Problem

You need to fetch and parse a HTML document from the web, and find data within it (screen scraping).

Solution

Use the Jsoup.connect(String url) method:

  1. Document doc = Jsoup.connect("http://example.com/").get();
  2. String title = doc.title();

Description

The connect(String url) method creates a new Connection, and get() fetches and parses a HTML file. If an error occurs whilst fetching the URL, it will throw an IOException, which you should handle appropriately.

The Connection interface is designed for method chaining to build specific requests:

  1. Document doc = Jsoup.connect("http://example.com")
  2.   .data("query", "Java")
  3.   .userAgent("Mozilla")
  4.   .cookie("auth", "token")
  5.   .timeout(3000)
  6.   .post();

This method only suports web URLs (http and https protocols); if you need to load from a file, use the parse(File in, String charsetName) method instead.