Enterprise Java

Reduce Bot Detection for Selenium in Spring Boot Applications

Websites today employ a variety of methods to detect automated tools such as Selenium. They inspect browser fingerprints, monitor navigation timing, detect WebDriver flags, analyze scroll and mouse interactions, and even examine network behavior to determine whether a real user is behind the session. When building Spring Boot applications that rely on Selenium for automation or testing, bot detection can block pages, trigger CAPTCHAs, or return incomplete content.

This article explains how to configure Selenium in undetected mode within a Spring Boot application, set up proper headers and flags, and run real-world tests to confirm that your automated browser behaves more like a typical user.

1. Understanding Why Sites Detect Bots

Websites often detect and flag automation tools because of identifiable patterns in how browsers behave. Common triggers include WebDriver fingerprints, unusual navigator properties, missing user interaction events, non-human scrolling or timing patterns, headless browser signatures, irregular request headers or JavaScript execution timing, rapid or repetitive requests, and anomalies from proxies or VPNs.

The purpose of addressing these signals is not to bypass security for malicious scraping, but to ensure that automated tests and legitimate workflows using Selenium behave like real users. By mimicking natural browsing behavior, you can reduce unnecessary CAPTCHA interruptions and improve the reliability of automation tasks.

2. Integrating Selenium into a Spring Boot Project

Before configuring the browser, we need to define our Maven dependencies.

        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-java</artifactId>
            <version>4.38.0</version>
        </dependency>
        <dependency>
            <groupId>io.github.bonigarcia</groupId>
            <artifactId>webdrivermanager</artifactId>
            <version>6.3.3</version>
        </dependency>

These dependencies provide everything needed to run Selenium inside Spring Boot. WebDriverManager ensures the correct ChromeDriver version is downloaded automatically, while Selenium gives us the API to launch Chrome in undetected mode.

3. Configuring Undetected Selenium in Spring Boot

Below is the configuration that enables an undetected Chrome session. It disables the automation flag, removes WebDriver signatures, and uses the new Chrome headless mode (which is less detectable).

@Configuration
public class SeleniumConfig {

    @Bean
    public WebDriver webDriver() {

        WebDriverManager.chromedriver().setup();

        ChromeOptions options = new ChromeOptions();

        // Remove automation flags
        options.setExperimentalOption("excludeSwitches",
                Collections.singletonList("enable-automation"));
        options.setExperimentalOption("useAutomationExtension", false);

        // Disable blink automation flag
        options.addArguments("--disable-blink-features=AutomationControlled");

        // New undetected headless mode
        options.addArguments("--headless=new");

        // Realistic user-agent
        options.addArguments("user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64)");

        ChromeDriver driver = new ChromeDriver(options);

        // CDP commands to reduce WebDriver fingerprints
        driver.executeCdpCommand("Page.addScriptToEvaluateOnNewDocument", Map.of(
                "source", "Object.defineProperty(navigator, 'webdriver', {get: () => undefined})"
        ));

        return driver;
    }
}

This configuration creates an undetected Chrome instance by removing WebDriver-related switches, masking automation properties, and enabling realistic browser behavior. The line options.addArguments("--disable-blink-features=AutomationControlled") prevents Chrome from exposing internal flags that indicate automated control. Combined with the removal of the "enable-automation" flag and CDP commands (like overriding navigator.webdriver), these adjustments significantly reduce Selenium’s detectable fingerprint.

Service to Load a URL

@Service
public class BrowserService {

    private final WebDriver driver;

    public BrowserService(WebDriver driver) {
        this.driver = driver;
    }

    public String loadPage(String url) {
        driver.get(url);

        // Simulate small human-like delay
        try {
            Thread.sleep(1200);
        } catch (Exception ignored) {
        }

        return driver.getPageSource();
    }
}

This service loads a target URL using the undetected Selenium browser instance. A short delay allows dynamic content to fully render before capturing the HTML. The returned page source helps verify whether the website loaded normally or triggered anti-bot mechanisms.

Controller Endpoint

@RestController
@RequestMapping("/browser")
public class BrowserController {

    private final BrowserService browserService;

    public BrowserController(BrowserService browserService) {
        this.browserService = browserService;
    }

    @GetMapping
    public ResponseEntity<String> open(@RequestParam String url) {
        String html = browserService.loadPage(url);
        return ResponseEntity.ok(html);
    }
}

The /browser endpoint takes a URL parameter and returns the HTML response obtained by Selenium. This allows you to test how the undetected browser behaves from any REST client, such as curl or Postman.

3.1 Testing Against Bot Detection Websites

To confirm Selenium is not flagged as a bot, test using sites that detect automation.

WebDriver Detection

This test checks whether the browser is detected as headless or automated by scanning for common WebDriver flags.

curl "http://localhost:8080/browser?url=https://arh.antoinevastel.com/bots/areyouheadless"

General Automation Detection

This test evaluates additional automation signals, such as missing user interactions, navigator properties, and typical automation behaviour.

curl "http://localhost:8080/browser?url=https://fingerprint-scan.com/"

4. Conclusion

Using Selenium in undetected mode within a Spring Boot application allows automated workflows to operate more reliably, avoiding unnecessary blocking triggered by bot detection systems. Through careful configuration, removing WebDriver signals, applying realistic browser behaviour, and using updated Chrome flags, the resulting automation behaves more naturally and returns pages more consistently.

5. Download the Source Code

This article focused on how to avoid bot detection using Selenium in Spring Boot.

Download
You can download the full source code of this example here: how to avoid bot detection using Selenium in Spring Boot

Omozegie Aziegbe

Omos Aziegbe is a technical writer and web/application developer with a BSc in Computer Science and Software Engineering from the University of Bedfordshire. Specializing in Java enterprise applications with the Jakarta EE framework, Omos also works with HTML5, CSS, and JavaScript for web development. As a freelance web developer, Omos combines technical expertise with research and writing on topics such as software engineering, programming, web application development, computer science, and technology.
Subscribe
Notify of
guest

This site uses Akismet to reduce spam. Learn how your comment data is processed.

0 Comments
Oldest
Newest Most Voted
Back to top button