获取指定页面的HTML源码

最新推荐文章于 2022-10-18 12:00:08 发布

原创最新推荐文章于 2022-10-18 12:00:08 发布 · 3.7k 阅读

1 ·

本内容遵循CC 4.0 BY-SA版权协议

标签

#html #string #stream #windows #encoding

点滴积累专栏收录该内容

46 篇文章

订阅专栏

本文介绍了两种通过C#编程语言获取指定网页HTML源码的方法。一种是使用简单的WebRequest和WebResponse对象来抓取网页内容，另一种则加入了更多的异常处理及自定义User-Agent头，以提高抓取的成功率。

/// <summary> /// 获取指定页面的HTML源码 /// </summary> /// <param name="PageURL">目标网页地址</param> /// <param name="Encode">编码方式</param> /// <returns>目标网页源文件的HTML字符串</returns> public string GetPageCode(string PageURL, Encoding Encode) { //存放目标网页的html string strHtml = string.Empty; HttpWebResponse wresp; try { //连接到目标网页 WebRequest wreq = WebRequest.Create(PageURL); wresp = (HttpWebResponse)wreq.GetResponse(); } catch (WebException ex) { wresp = (HttpWebResponse)ex.Response; } //采用流读取，并确定编码方式 StreamReader objReader = new StreamReader(wresp.GetResponseStream(), Encode); return strHtml = objReader.ReadToEnd(); }

调用：

string PageCode = GetPageCode(url, Encoding.GetEncoding("gb2312"));

于2010-09-06补上net_lover【孟子E章】方法：

//获取指定页面的源代码 public String GetPageCode(String PageURL, String Charset) { String strHtml = ""; try { //连接到目标网页 HttpWebRequest wreq = (HttpWebRequest)WebRequest.Create(PageURL); wreq.Method = "GET"; wreq.UserAgent = "Mozilla/5.0 (Windows; U; Windows NT 6.1; zh-CN; rv:1.9.2.8) Gecko/20100722 Firefox/3.6.8"; HttpWebResponse wresp = (HttpWebResponse)wreq.GetResponse(); //采用流读取，并确定编码方式 Stream s = wresp.GetResponseStream(); StreamReader objReader = new StreamReader(s, System.Text.Encoding.GetEncoding(Charset)); strHtml = objReader.ReadToEnd(); objReader.Close(); return strHtml; } catch(WebException ex) { HttpWebResponse res = ex.Response as HttpWebResponse; if (res.StatusCode == HttpStatusCode.InternalServerError) { Stream s = res.GetResponseStream(); StreamReader objReader = new StreamReader(s, System.Text.Encoding.GetEncoding(Charset)); strHtml = objReader.ReadToEnd(); objReader.Close(); } else { strHtml = ex.Message; } return strHtml; } }

原帖地址：http://topic.csdn.net/u/20100904/09/0277af79-8e7a-43ca-ae25-538f5ac46b35.html