Example program: list links

This example program demonstrates how to fetch a page from a URL; extract links, images, and other pointers; and examine their URLs and text.

Specify the URL to fetch as the program's sole argument.

  1. package org.jsoup.examples;
  3. import org.jsoup.Jsoup;
  4. import org.jsoup.helper.Validate;
  5. import org.jsoup.nodes.Document;
  6. import org.jsoup.nodes.Element;
  7. import org.jsoup.select.Elements;
  9. import java.io.IOException;
  11. /**
  12.  * Example program to list links from a URL.
  13.  */
  14. public class ListLinks {
  15.     public static void main(String[] args) throws IOException {
  16.         Validate.isTrue(args.length == 1, "usage: supply url to fetch");
  17.         String url = args[0];
  18.         print("Fetching %s...", url);
  20.         Document doc = Jsoup.connect(url).get();
  21.         Elements links = doc.select("a[href]");
  22.         Elements media = doc.select("[src]");
  23.         Elements imports = doc.select("link[href]");
  25.         print("\nMedia: (%d)", media.size());
  26.         for (Element src : media) {
  27.             if (src.normalName().equals("img"))
  28.                 print(" * %s: <%s> %sx%s (%s)",
  29.                         src.tagName(), src.attr("abs:src"), src.attr("width"), src.attr("height"),
  30.                         trim(src.attr("alt"), 20));
  31.             else
  32.                 print(" * %s: <%s>", src.tagName(), src.attr("abs:src"));
  33.         }
  35.         print("\nImports: (%d)", imports.size());
  36.         for (Element link : imports) {
  37.             print(" * %s <%s> (%s)", link.tagName(),link.attr("abs:href"), link.attr("rel"));
  38.         }
  40.         print("\nLinks: (%d)", links.size());
  41.         for (Element link : links) {
  42.             print(" * a: <%s>  (%s)", link.attr("abs:href"), trim(link.text(), 35));
  43.         }
  44.     }
  46.     private static void print(String msg, Object... args) {
  47.         System.out.println(String.format(msg, args));
  48.     }
  50.     private static String trim(String s, int width) {
  51.         if (s.length() > width)
  52.             return s.substring(0, width-1) + ".";
  53.         else
  54.             return s;
  55.     }
  56. }
  57. org/jsoup/examples/ListLinks.java

Example output (trimmed)

  1. Fetching http://news.ycombinator.com/...
  3. Media: (38)
  4.  * img: <http://ycombinator.com/images/y18.gif> 18x18 ()
  5.  * img: <http://ycombinator.com/images/s.gif> 10x1 ()
  6.  * img: <http://ycombinator.com/images/grayarrow.gif> x ()
  7.  * img: <http://ycombinator.com/images/s.gif> 0x10 ()
  8.  * script: <http://www.co2stats.com/propres.php?s=1138>
  9.  * img: <http://ycombinator.com/images/s.gif> 15x1 ()
  10.  * img: <http://ycombinator.com/images/hnsearch.png> x ()
  11.  * img: <http://ycombinator.com/images/s.gif> 25x1 ()
  12.  * img: <http://mixpanel.com/site_media/images/mixpanel_partner_logo_borderless.gif> x (Analytics by Mixpan.)
  14. Imports: (2)
  15.  * link <http://ycombinator.com/news.css> (stylesheet)
  16.  * link <http://ycombinator.com/favicon.ico> (shortcut icon)
  18. Links: (141)
  19.  * a: <http://ycombinator.com>  ()
  20.  * a: <http://news.ycombinator.com/news>  (Hacker News)
  21.  * a: <http://news.ycombinator.com/newest>  (new)
  22.  * a: <http://news.ycombinator.com/newcomments>  (comments)
  23.  * a: <http://news.ycombinator.com/leaders>  (leaders)
  24.  * a: <http://news.ycombinator.com/jobs>  (jobs)
  25.  * a: <http://news.ycombinator.com/submit>  (submit)
  26.  * a: <http://news.ycombinator.com/x?fnid=JKhQjfU7gW>  (login)
  27.  * a: <http://news.ycombinator.com/vote?for=1094578&dir=up&whence=%6e%65%77%73>  ()
  28.  * a: <http://www.readwriteweb.com/archives/facebook_gets_faster_debuts_homegrown_php_compiler.php?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+readwriteweb+%28ReadWriteWeb%29&utm_content=Twitter>  (Facebook speeds up PHP)
  29.  * a: <http://news.ycombinator.com/user?id=mcxx>  (mcxx)
  30.  * a: <http://news.ycombinator.com/item?id=1094578>  (9 comments)
  31.  * a: <http://news.ycombinator.com/vote?for=1094649&dir=up&whence=%6e%65%77%73>  ()
  32.  * a: <http://groups.google.com/group/django-developers/msg/a65fbbc8effcd914>  ("Tough. Django produces XHTML.")
  33.  * a: <http://news.ycombinator.com/user?id=andybak>  (andybak)
  34.  * a: <http://news.ycombinator.com/item?id=1094649>  (3 comments)
  35.  * a: <http://news.ycombinator.com/vote?for=1093927&dir=up&whence=%6e%65%77%73>  ()
  36.  * a: <http://news.ycombinator.com/x?fnid=p2sdPLE7Ce>  (More)
  37.  * a: <http://news.ycombinator.com/lists>  (Lists)
  38.  * a: <http://news.ycombinator.com/rss>  (RSS)
  39.  * a: <http://ycombinator.com/bookmarklet.html>  (Bookmarklet)
  40.  * a: <http://ycombinator.com/newsguidelines.html>  (Guidelines)
  41.  * a: <http://ycombinator.com/newsfaq.html>  (FAQ)
  42.  * a: <http://ycombinator.com/newsnews.html>  (News News)
  43.  * a: <http://news.ycombinator.com/item?id=363>  (Feature Requests)
  44.  * a: <http://ycombinator.com>  (Y Combinator)
  45.  * a: <http://ycombinator.com/w2010.html>  (Apply)
  46.  * a: <http://ycombinator.com/lib.html>  (Library)
  47.  * a: <http://www.webmynd.com/html/hackernews.html>  ()
  48.  * a: <http://mixpanel.com/?from=yc>  ()