Apache HttpClient配置使用详解

一、Apache HttpClient 简介

Apache HttpClient 是 Java 上最流行的 HTTP 客户端库之一，支持 GET、POST、PUT、DELETE 等各种 HTTP 操作，且易于扩展和配置。

常用版本为 org.apache.httpcomponents:httpclient（4.x）或 org.apache.httpcomponents.client5:httpclient5（5.x）。下面以 4.x 为主，5.x 有类似用法。

---

二、依赖引入

Maven：

<dependency>
    <groupId>org.apache.httpcomponents</groupId>
    <artifactId>httpclient</artifactId>
    <version>4.5.14</version>
</dependency>

---

三、基本使用

import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;

public class HttpClientDemo {
    public static void main(String[] args) throws Exception {
        try (CloseableHttpClient client = HttpClients.createDefault()) {
            HttpGet request = new HttpGet("https://www.example.com");
            try (CloseableHttpResponse response = client.execute(request)) {
                System.out.println(response.getStatusLine());
            }
        }
    }
}

---

四、常用配置详解

1. 超时设置

包括连接超时、请求超时、socket超时：

import org.apache.http.client.config.RequestConfig;

RequestConfig config = RequestConfig.custom()
    .setConnectTimeout(5000)    // 连接超时
    .setSocketTimeout(10000)    // 读取超时
    .setConnectionRequestTimeout(2000) // 从连接池获取连接超时
    .build();

HttpGet request = new HttpGet("https://www.example.com");
request.setConfig(config);

2. 代理设置

import org.apache.http.HttpHost;

HttpHost proxy = new HttpHost("proxy.example.com", 8080);
RequestConfig config = RequestConfig.custom()
    .setProxy(proxy)
    .build();
request.setConfig(config);

3. 请求头设置

request.setHeader("User-Agent", "Apache-HttpClient/4.5.14");
request.setHeader("Accept", "application/json");

4. Cookie 管理

import org.apache.http.impl.client.BasicCookieStore;
import org.apache.http.impl.client.HttpClientBuilder;

BasicCookieStore cookieStore = new BasicCookieStore();
CloseableHttpClient client = HttpClientBuilder.create()
    .setDefaultCookieStore(cookieStore)
    .build();

5. 连接池配置（推荐用于高并发）

import org.apache.http.impl.conn.PoolingHttpClientConnectionManager;

PoolingHttpClientConnectionManager connManager = new PoolingHttpClientConnectionManager();
connManager.setMaxTotal(100); // 最大连接数
connManager.setDefaultMaxPerRoute(20); // 每个路由最大连接数

CloseableHttpClient client = HttpClients.custom()
    .setConnectionManager(connManager)
    .build();

6. SSL证书信任（跳过验证，仅测试用）

import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
import org.apache.http.conn.ssl.TrustAllStrategy;
import org.apache.http.ssl.SSLContextBuilder;

SSLConnectionSocketFactory sslsf = new SSLConnectionSocketFactory(
    SSLContextBuilder.create().loadTrustMaterial(new TrustAllStrategy()).build(),
    SSLConnectionSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER);

CloseableHttpClient client = HttpClients.custom()
    .setSSLSocketFactory(sslsf)
    .build();

---

五、发送POST请求（含JSON）

import org.apache.http.entity.StringEntity;
import org.apache.http.client.methods.HttpPost;

HttpPost post = new HttpPost("https://api.example.com/data");
post.setHeader("Content-Type", "application/json");
post.setEntity(new StringEntity("{\"name\":\"value\"}", "UTF-8"));

try (CloseableHttpResponse response = client.execute(post)) {
    System.out.println(EntityUtils.toString(response.getEntity()));
}

---

六、重试机制

import org.apache.http.impl.client.DefaultHttpRequestRetryHandler;

CloseableHttpClient client = HttpClients.custom()
    .setRetryHandler(new DefaultHttpRequestRetryHandler(3, true))
    .build();

---

七、完整配置示例

PoolingHttpClientConnectionManager connManager = new PoolingHttpClientConnectionManager();
connManager.setMaxTotal(200);
connManager.setDefaultMaxPerRoute(50);

RequestConfig config = RequestConfig.custom()
    .setConnectTimeout(5000)
    .setSocketTimeout(10000)
    .setConnectionRequestTimeout(2000)
    .build();

BasicCookieStore cookieStore = new BasicCookieStore();

CloseableHttpClient client = HttpClients.custom()
    .setConnectionManager(connManager)
    .setDefaultCookieStore(cookieStore)
    .setRetryHandler(new DefaultHttpRequestRetryHandler(3, true))
    .build();

HttpGet get = new HttpGet("https://www.example.com/api");
get.setConfig(config);
get.setHeader("Accept", "application/json");

try (CloseableHttpResponse response = client.execute(get)) {
    System.out.println(EntityUtils.toString(response.getEntity()));
}

---

八、最佳实践与注意事项

1. 连接池复用：高并发场景下务必启用连接池，不要频繁 new HttpClient。
2. 关闭资源：用完响应和客户端要关闭（try-with-resources）。
3. 合理设置超时：避免因网络故障导致线程阻塞。
4. 线程安全：PoolingHttpClientConnectionManager 是线程安全的。
5. 异常处理：捕获并处理 IOException、HttpException 等。
6. 日志调试：可用 SLF4J 等日志框架输出请求/响应调试信息。

九、异步请求（HttpAsyncClient）

如果你需要非阻塞式的 HTTP 请求，可以使用 HttpAsyncClient。

依赖：

<dependency>
    <groupId>org.apache.httpcomponents</groupId>
    <artifactId>httpasyncclient</artifactId>
    <version>4.1.4</version>
</dependency>

示例：

import org.apache.http.impl.nio.client.CloseableHttpAsyncClient;
import org.apache.http.impl.nio.client.HttpAsyncClients;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.concurrent.FutureCallback;

CloseableHttpAsyncClient asyncClient = HttpAsyncClients.createDefault();
asyncClient.start();

HttpGet request = new HttpGet("https://www.example.com");
asyncClient.execute(request, new FutureCallback<HttpResponse>() {
    public void completed(HttpResponse response) {
        System.out.println("Response: " + response.getStatusLine());
    }
    public void failed(Exception ex) {
        System.out.println("Failed: " + ex.getMessage());
    }
    public void cancelled() {
        System.out.println("Cancelled");
    }
});

// 关闭客户端
asyncClient.close();

---

十、认证机制

1. Basic Auth

import org.apache.http.auth.UsernamePasswordCredentials;
import org.apache.http.impl.client.BasicCredentialsProvider;
import org.apache.http.auth.AuthScope;

BasicCredentialsProvider credsProvider = new BasicCredentialsProvider();
credsProvider.setCredentials(
    AuthScope.ANY,
    new UsernamePasswordCredentials("user", "password")
);

CloseableHttpClient client = HttpClients.custom()
    .setDefaultCredentialsProvider(credsProvider)
    .build();

2. Bearer Token（如 OAuth2）

HttpGet get = new HttpGet("https://api.example.com");
get.setHeader("Authorization", "Bearer your_token");

3. Digest Auth / NTLM / Kerberos

可以参考官方文档，配置 CredentialsProvider 和 AuthScheme。

---

十一、文件上传（多部分表单）

import org.apache.http.entity.mime.MultipartEntityBuilder;
import org.apache.http.entity.ContentType;
import org.apache.http.client.methods.HttpPost;

HttpPost post = new HttpPost("https://api.example.com/upload");
MultipartEntityBuilder builder = MultipartEntityBuilder.create();
builder.addTextBody("description", "test file", ContentType.TEXT_PLAIN);
builder.addBinaryBody("file", new File("test.jpg"), ContentType.APPLICATION_OCTET_STREAM, "test.jpg");
post.setEntity(builder.build());

try (CloseableHttpResponse response = client.execute(post)) {
    System.out.println(EntityUtils.toString(response.getEntity()));
}

---

十二、文件下载（流式读取）

HttpGet get = new HttpGet("https://example.com/file.zip");
try (CloseableHttpResponse response = client.execute(get);
     InputStream in = response.getEntity().getContent();
     FileOutputStream out = new FileOutputStream("file.zip")) {
    byte[] buffer = new byte[4096];
    int bytesRead;
    while ((bytesRead = in.read(buffer)) != -1) {
        out.write(buffer, 0, bytesRead);
    }
}

---

十三、连接管理与关闭

• 连接池推荐使用 PoolingHttpClientConnectionManager。
• 客户端和响应都要关闭，避免资源泄漏。
• 可以定期清理过期连接：

  connManager.closeExpiredConnections();
  connManager.closeIdleConnections(30, TimeUnit.SECONDS);
  

---

十四、日志与调试

可以通过配置日志框架（如 Log4j、SLF4J）输出 HttpClient 的调试日志：

# log4j.properties 示例
log4j.logger.org.apache.http=DEBUG
log4j.logger.org.apache.http.wire=DEBUG

这样可以看到请求、响应、头信息等详细内容。

---

十五、常见问题与解决方案

1. 端口耗尽/连接泄漏：
   • 使用连接池，及时关闭响应和客户端。
2. SSL证书错误：
   • 测试环境可用 TrustAllStrategy 跳过验证，生产环境需正确配置证书。
3. 长时间阻塞：
   • 合理设置超时参数，避免无限等待。
4. Cookie/Session丢失：
   • 使用 CookieStore 维持会话。
5. 并发问题：
   • HttpClient、PoolingHttpClientConnectionManager 都是线程安全的，可以多线程复用。

---

十六、升级到 HttpClient 5.x

HttpClient 5.x 提供了更好的 HTTP/2 支持、更现代的 API，配置方式类似但更灵活。推荐新项目使用 5.x。

文档：
https://hc.apache.org/httpcomponents-client-5.2.x/index.html

---

十七、其他高级特性

• 请求拦截器/响应拦截器（HttpRequestInterceptor/HttpResponseInterceptor）
• 自定义重试策略
• 异步流式处理
• 连接路由和 DNS 配置
• 代理认证和多级代理

十八、请求与响应拦截器

拦截器可以实现请求/响应的统一处理，比如自动加头、日志、签名等。

1. 请求拦截器

import org.apache.http.HttpRequest;
import org.apache.http.HttpRequestInterceptor;
import org.apache.http.protocol.HttpContext;

HttpRequestInterceptor requestInterceptor = new HttpRequestInterceptor() {
    public void process(HttpRequest request, HttpContext context) {
        request.addHeader("X-Custom-Header", "value");
        // 可做日志、签名等
    }
};

CloseableHttpClient client = HttpClients.custom()
    .addInterceptorFirst(requestInterceptor)
    .build();

2. 响应拦截器

import org.apache.http.HttpResponse;
import org.apache.http.HttpResponseInterceptor;

HttpResponseInterceptor responseInterceptor = new HttpResponseInterceptor() {
    public void process(HttpResponse response, HttpContext context) {
        // 统一处理响应，比如日志、解密等
        System.out.println("Response status: " + response.getStatusLine());
    }
};

CloseableHttpClient client = HttpClients.custom()
    .addInterceptorLast(responseInterceptor)
    .build();

---

十九、自定义重试策略

可以根据异常类型和请求幂等性定制重试策略。

import org.apache.http.client.HttpRequestRetryHandler;
import org.apache.http.protocol.HttpContext;
import java.io.IOException;

HttpRequestRetryHandler retryHandler = (exception, executionCount, context) -> {
    if (executionCount >= 5) return false; // 最大重试5次
    if (exception instanceof IOException) return true; // 网络异常重试
    return false;
};

CloseableHttpClient client = HttpClients.custom()
    .setRetryHandler(retryHandler)
    .build();

---

二十、连接池监控与管理

可以定期清理连接池中的过期和空闲连接，避免资源泄漏。

PoolingHttpClientConnectionManager connManager = new PoolingHttpClientConnectionManager();
// ...配置连接池
// 定期清理线程
new Thread(() -> {
    while (true) {
        try {
            Thread.sleep(30000);
            connManager.closeExpiredConnections();
            connManager.closeIdleConnections(30, TimeUnit.SECONDS);
        } catch (InterruptedException e) {
            break;
        }
    }
}).start();

---

二十一、自定义DNS解析

HttpClient 4.x/5.x 都支持自定义 DNS 解析（如本地hosts优先、特殊域名解析）。

import org.apache.http.conn.DnsResolver;

DnsResolver dnsResolver = new DnsResolver() {
    public InetAddress[] resolve(String host) throws UnknownHostException {
        if ("special.example.com".equals(host)) {
            return new InetAddress[] { InetAddress.getByName("1.2.3.4") };
        }
        return InetAddress.getAllByName(host);
    }
};

PoolingHttpClientConnectionManager connManager =
    new PoolingHttpClientConnectionManager(
        RegistryBuilder.<ConnectionSocketFactory>create()
            .register("http", PlainConnectionSocketFactory.INSTANCE)
            .register("https", SSLConnectionSocketFactory.getSocketFactory())
            .build(),
        null, null, dnsResolver, 5000, TimeUnit.MILLISECONDS
    );

---

二十二、高级代理与认证

支持多级代理、代理认证（如公司内网/SSO等）。

HttpHost proxy = new HttpHost("proxy.example.com", 8080);
CredentialsProvider credsProvider = new BasicCredentialsProvider();
credsProvider.setCredentials(
    new AuthScope("proxy.example.com", 8080),
    new UsernamePasswordCredentials("user", "password")
);

CloseableHttpClient client = HttpClients.custom()
    .setDefaultCredentialsProvider(credsProvider)
    .setProxy(proxy)
    .build();

---

二十三、HTTP/2 支持（HttpClient 5.x）

HttpClient 5.x 支持 HTTP/2 协议，提升性能。

依赖：

<dependency>
    <groupId>org.apache.httpcomponents.client5</groupId>
    <artifactId>httpclient5</artifactId>
    <version>5.2.1</version>
</dependency>

代码示例：

import org.apache.hc.client5.http.impl.async.CloseableHttpAsyncClient;
import org.apache.hc.client5.http.impl.async.HttpAsyncClients;
import org.apache.hc.core5.http.nio.support.BasicRequestProducer;
import org.apache.hc.core5.http.nio.support.BasicResponseConsumer;
import org.apache.hc.core5.http.message.BasicHttpRequest;

CloseableHttpAsyncClient client = HttpAsyncClients.createHttp2Default();
client.start();

BasicHttpRequest request = new BasicHttpRequest("GET", "https://http2.example.com/");
client.execute(
    BasicRequestProducer.create(request),
    BasicResponseConsumer.create(),
    new FutureCallback<Message<HttpResponse, String>>() {
        public void completed(Message<HttpResponse, String> message) {
            System.out.println("HTTP/2 response: " + message.getBody());
        }
        public void failed(Exception ex) { System.out.println("Failed: " + ex.getMessage()); }
        public void cancelled() { System.out.println("Cancelled"); }
    }
);

---

二十四、常见应用场景代码片段

1. 批量并发请求

ExecutorService pool = Executors.newFixedThreadPool(10);
for (String url : urls) {
    pool.submit(() -> {
        HttpGet get = new HttpGet(url);
        try (CloseableHttpResponse resp = client.execute(get)) {
            System.out.println(EntityUtils.toString(resp.getEntity()));
        } catch (Exception e) {
            e.printStackTrace();
        }
    });
}
pool.shutdown();

2. 下载大文件断点续传

HttpGet get = new HttpGet("https://example.com/bigfile.zip");
get.setHeader("Range", "bytes=1000000-"); // 断点续传

3. 上传大文件分片

使用 MultipartEntityBuilder，分多次上传分片。

---

二十五、常用调试技巧

• 设置日志级别为 DEBUG，查看请求、响应、连接池状态。
• 使用 Wireshark 或 Fiddler 抓包分析。
• 结合 JMX 监控连接池、线程池状态。