If you build sales tools, recruiting pipelines, or enrichment workflows, you need LinkedIn profile data. Name, title, company, experience, education. The profile is the starting point for almost every B2B data pipeline.
The problem: LinkedIn does not offer a public API for profile data. So developers build scrapers. Most of them break. Here are 3 ways to scrape LinkedIn profiles with Python, ordered from most painful to most practical.
Method 1: DIY with Selenium or Playwright
The first thing most developers try. Spin up a headless browser, log in to LinkedIn, navigate to a profile page, and parse the HTML.
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.options import Options
import time
options = Options()
options.add_argument("--headless")
driver = webdriver.Chrome(options=options)
# Log in
driver.get("https://www.linkedin.com/login")
driver.find_element(By.ID, "username").send_keys("your@email.com")
driver.find_element(By.ID, "password").send_keys("your_password")
driver.find_element(By.CSS_SELECTOR, "button[type='submit']").click()
time.sleep(3)
# Navigate to profile
driver.get("https://www.linkedin.com/in/johndoe")
time.sleep(5)
# Parse profile data
name = driver.find_element(By.CSS_SELECTOR, "h1").text
headline = driver.find_element(By.CSS_SELECTOR, ".text-body-medium").text
print(f"Name: {name}")
print(f"Headline: {headline}")
driver.quit()
This works once. Maybe twice. Then reality sets in.
LinkedIn actively detects browser automation. After a handful of requests, you will hit CAPTCHAs, login challenges, and eventually account bans. The HTML structure changes without notice, breaking your selectors. You need to manage sessions, rotate proxies, handle JavaScript rendering, and deal with rate limiting. Every week, something new breaks.
Verdict: Works for 10 profiles. Breaks at 100.
Method 2: Browser Extensions (PhantomBuster, LinkedIn Helper)
Cloud-based browser automation tools like PhantomBuster, LinkedIn Helper, and Dux-Soup run a real browser session in the cloud. You give them your LinkedIn session cookie, configure a workflow, and they scrape profiles on a schedule.
The user experience is better than DIY, but these tools are built for sales teams clicking buttons, not for developers building pipelines.
- No programmatic access. There is no API. You configure workflows through a web UI and export results as CSV.
- Pricing does not scale. PhantomBuster starts at $56/month for 20 hours of execution time. At volume, you are paying $0.14 or more per profile.
- Pipeline integration is manual. Getting data from these tools into your codebase means downloading CSVs, setting up webhooks, or writing brittle integrations around their UI.
Verdict: Fine for manual prospecting. Not for automated pipelines.
Method 3: REST API (ScrapeLinkedIn.com)
The approach that actually works at scale. Send a LinkedIn URL to an API endpoint. Get back structured JSON with the profile data. No browser, no sessions, no maintenance.
import requests
import time
API_KEY = "li_live_YOUR_KEY"
BASE = "https://api.scrapelinkedin.com/api/v1"
headers = {"X-API-Key": API_KEY}
# Submit a scrape request
response = requests.post(
f"{BASE}/scrape",
json={"linkedin_url": "https://linkedin.com/in/johndoe"},
headers=headers
)
profile_id = response.json()["data"]["id"]
# Poll for result (typically 20-30 seconds)
time.sleep(30)
result = requests.get(
f"{BASE}/scrape/{profile_id}",
headers=headers
).json()
print(result["data"]["profile_data"])
That is the entire integration. No browser dependencies. No session management. No HTML parsing. Just HTTP requests and JSON responses.
Batch scraping
For larger jobs, the batch endpoint accepts up to 1,000 URLs in a single request.
import requests
import time
API_KEY = "li_live_YOUR_KEY"
BASE = "https://api.scrapelinkedin.com/api/v1"
headers = {"X-API-Key": API_KEY}
# Submit batch
urls = [
"https://linkedin.com/in/johndoe",
"https://linkedin.com/in/janedoe",
"https://linkedin.com/in/bobsmith",
# ... up to 1,000 URLs
]
batch = requests.post(
f"{BASE}/scrape/batch",
json={"linkedin_urls": urls},
headers=headers
).json()
batch_id = batch["data"]["batch_id"]
# Poll until complete
while True:
status = requests.get(
f"{BASE}/scrape/batch/{batch_id}",
headers=headers
).json()
if status["data"]["status"] == "completed":
for profile in status["data"]["results"]:
print(profile["profile_data"]["full_name"])
break
print(f"Progress: {status['data']['completed']}/{status['data']['total']}")
time.sleep(10)
Pricing
$0.01 per profile. Failed lookups are refunded automatically. You get 5 free credits on signup, no credit card required. No monthly commitment. Buy credits when you need them.
Verdict: The right choice for developers building data pipelines.
Quick Comparison
| Selenium | Extensions | API | |
|---|---|---|---|
| Setup time | Hours | Minutes | Seconds |
| Cost per profile | Free* | $0.14+ | $0.01 |
| Scales to 1,000+ | No | Manual | Yes |
| Programmatic | Yes | No | Yes |
| Maintenance | High | None | None |
*Free but costs engineering time and banned accounts.
Getting Started
- Register at the API docs. You get 5 free credits immediately.
- Verify your email with the 6-digit code sent to your inbox.
- Scrape with your API key.
One-liner to test it:
curl -X POST https://api.scrapelinkedin.com/api/v1/scrape \
-H "Content-Type: application/json" \
-H "X-API-Key: li_live_YOUR_KEY" \
-d '{"linkedin_url": "https://linkedin.com/in/johndoe"}'
Full API reference: API documentation.
Try it free. 5 profiles, no credit card.
Register, verify your email, and start scraping in under a minute.
Get Your API Key