Erlang Central

Read Web Page

From ErlangCentral Wiki

Problem

You want to get the contents of a web page. Output are the list of lines from the web page.

Solution 1

For a built in solution use the httpc module.


inets:start().
{ok, {{Version, 200, ReasonPhrase}, Headers, Body}} =
httpc:request(get, {"http://www.google.com", []}, [], []).

Solution 2

The ibrowse package is not an intrinsic erlang library; it can be obtained from the erlang user contribs section.


ibrowse:start().
 {ok, _StatusCode, _Headers, WebPageText} = ibrowse:send_req("http://www.google.com", [], get),
     Lines = string:tokens(WebPageText, "\n").