Selenium Integration
Use Enterprise Proxy Network with Selenium for web automation and testing
Basic Setup
Selenium is a popular framework for automating web browsers. It supports multiple browsers and programming languages, making it ideal for web scraping, testing, and automation tasks.
Installation
# Python
pip install selenium
# Node.js
npm install selenium-webdriver
# Java
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-java</artifactId>
<version>4.15.0</version>
</dependency>
WebDriver Setup
Download the appropriate WebDriver for your browser:
Chrome
ChromeDriver
Firefox
GeckoDriver
Edge
EdgeDriver
Proxy Configuration
Python Example
from selenium import webdriver
from selenium.webdriver.common.proxy import Proxy, ProxyType
# Configure proxy
proxy = Proxy()
proxy.proxy_type = ProxyType.MANUAL
proxy.http_proxy = "http://us-http-cust_123:sk_live_123456@proxy.enterpriseproxynetwork.com:8080"
proxy.ssl_proxy = "http://us-https-cust_123:sk_live_123456@proxy.enterpriseproxynetwork.com:8443"
proxy.socks_proxy = "socks5://us-socks5-cust_123:sk_live_123456@proxy.enterpriseproxynetwork.com:1080"
# Create Chrome options
options = webdriver.ChromeOptions()
options.add_argument('--proxy-server=http://us-http-cust_123:sk_live_123456@proxy.enterpriseproxynetwork.com:8080')
# Launch browser with proxy
driver = webdriver.Chrome(options=options)
# Test proxy connection
driver.get("https://api.ipify.org?format=json")
print(driver.find_element_by_tag_name("body").text)
JavaScript/Node.js Example
const { Builder, By } = require('selenium-webdriver');
const chrome = require('selenium-webdriver/chrome');
// Configure proxy
let options = new chrome.Options();
options.addArguments('--proxy-server=http://us-http-cust_123:sk_live_123456@proxy.enterpriseproxynetwork.com:8080');
// Launch browser
let driver = await new Builder()
.forBrowser('chrome')
.setChromeOptions(options)
.build();
// Test connection
await driver.get('https://api.ipify.org?format=json');
let body = await driver.findElement(By.tagName('body'));
console.log(await body.getText());
Java Example
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;
public class SeleniumProxyExample {
public static void main(String[] args) {
// Configure proxy
ChromeOptions options = new ChromeOptions();
options.addArguments("--proxy-server=http://us-http-cust_123:sk_live_123456@proxy.enterpriseproxynetwork.com:8080");
// Launch browser
WebDriver driver = new ChromeDriver(options);
// Test connection
driver.get("https://api.ipify.org?format=json");
System.out.println(driver.getPageSource());
driver.quit();
}
}
Browser-Specific Configuration
Chrome/Chromium
# Chrome with SOCKS5 proxy
options.addArguments('--proxy-server=socks5://proxy.enterpriseproxynetwork.com:1080')
# For authentication, you'll need to handle it differently
# Option 1: Use browser extension
# Option 2: Use AutoIt or similar for auth dialog
# Option 3: Use --proxy-server with credentials in URL (limited support)
Firefox
from selenium.webdriver.firefox.options import Options
options = Options()
options.set_preference('network.proxy.type', 1)
options.set_preference('network.proxy.socks', 'proxy.enterpriseproxynetwork.com')
options.set_preference('network.proxy.socks_port', 1080)
options.set_preference('network.proxy.socks_username', 'us-socks5-cust_123')
options.set_preference('network.proxy.socks_password', 'sk_live_123456')
options.set_preference('network.proxy.socks_version', 5)
driver = webdriver.Firefox(options=options)
Edge
from selenium.webdriver.edge.options import Options
options = Options()
options.add_argument('--proxy-server=http://us-http-cust_123:sk_live_123456@proxy.enterpriseproxynetwork.com:8080')
driver = webdriver.Edge(options=options)
Performance Optimization
Headless Mode
Use headless mode for better performance and resource usage:
options.add_argument('--headless')
options.add_argument('--disable-gpu')
options.add_argument('--no-sandbox')
Resource Management
Disable unnecessary features to improve performance:
options.add_argument('--disable-dev-shm-usage')
options.add_argument('--disable-extensions')
options.add_argument('--disable-notifications')
Connection Pooling
# Reuse browser instances when possible
# Instead of creating new browser for each task
browser_pool = []
def get_browser():
if not browser_pool:
options = create_proxy_options()
browser = webdriver.Chrome(options=options)
browser_pool.append(browser)
return browser_pool.pop()
def return_browser(browser):
browser_pool.append(browser)
# Usage
browser = get_browser()
try:
# Do work
browser.get('https://example.com')
finally:
return_browser(browser)
Best Practices & Troubleshooting
Error Handling
from selenium.common.exceptions import WebDriverException
import time
def with_retry(func, max_retries=3):
for attempt in range(max_retries):
try:
return func()
except WebDriverException as e:
print(f"Attempt {attempt + 1} failed: {e}")
if attempt == max_retries - 1:
raise
time.sleep(2 ** attempt) # Exponential backoff
# Usage
def test_connection():
driver.get("https://api.ipify.org")
return driver.find_element_by_tag_name("body").text
ip = with_retry(test_connection)
Common Issues
Authentication Dialogs
Some browsers show authentication dialogs for proxy servers. Use browser extensions or automated tools like AutoIt to handle these dialogs.
Certificate Errors
For HTTPS proxies, you may encounter certificate errors. Add the proxy's CA certificate to your browser's trust store or use options to ignore SSL errors.
Timeout Issues
Increase timeout settings for proxy connections and implement proper retry logic with exponential backoff.
Testing Proxy Connection
# Simple test to verify proxy is working
driver.get("https://api.ipify.org?format=json")
ip_element = driver.find_element_by_tag_name("body")
print(f"Proxy IP: {ip_element.text}")
# Should display your proxy IP, not your real IP
# If it shows your real IP, proxy configuration failed
Was this helpful?
Help us improve our documentation