GoFuckYourself.com - Adult Webmaster Forum - View Single Post - Tech Claude Code research preview

JustBiz · 03-12-2025, 06:54 PM

Second thread I read all the way through on GFY in quite a while ...

I promised a friend I'd make him a scraper without charge many months ago. But have had no time. Seemed a good enough excuse right now to give Claude a quick try and only via the chat window.

I gave Claude an outline and, after asking not to interrupt til I finished, about eight bullet points to build it. It said I clearly had thought through the functionality very seriously (I hadn't), and said it should be ok to do using Python with libraries like BeautifulSoup for scraping and Pandas for data manipulation.

Few minutes later, I had a fully working scraper which rolled along fine first time in Python. Was gonna close the chat, thinking I'll have to make some sort of executable, cos no way friend is gonna use command line.

Then asked Claude: "Don't suppose you can make something in javascript with a HTML UI, providing we include a CORS proxy?"

He says he doesn't see why not, that it would be quite an elegant solution and easy for someone to run. We discuss a few limitations this way, if scraped site had infinite scrolling etc. But agree it's a decent enough trade-off.

I ask for some extra stuff in the HTML, including changing a very specific list of hardcoded URLs to be scraped to something that won't require me to update it for new sites or site changes. Just added a field to UI where user pastes snippet of code where what he needs scraped dwells. :=)

Couple more minutes and Claude delivers something my friend will be able to use, just by opening a webpage. Looks nice enough, tells him exactly what to paste where and deliver outputs which show what he scraped that's new, updates his existing lists, orders alphabetically in CSV and counts/displays totals.

As suspected, it doesn't work without CORS proxy, so he has a field to enter that too.

Instead of being something designed for his specific needs, he can now use it to scrape pretty much anything. If he can't find the shiny code arrow that says "Scrape here" in any source code, he can just ask and I can tell him without having to touch code again.

Has some limitations that python version doesn't. But pretty good, and will do more than my friend asked for, without me having to do anything but request a few things.

Another thumb trending up for Claude, I guess.

03-12-2025, 06:54 PM
JustBiz Confirmed User Industry Role: Join Date: Jun 2019 Posts: 2,113	Second thread I read all the way through on GFY in quite a while ... I promised a friend I'd make him a scraper without charge many months ago. But have had no time. Seemed a good enough excuse right now to give Claude a quick try and only via the chat window. I gave Claude an outline and, after asking not to interrupt til I finished, about eight bullet points to build it. It said I clearly had thought through the functionality very seriously (I hadn't), and said it should be ok to do using Python with libraries like BeautifulSoup for scraping and Pandas for data manipulation. Few minutes later, I had a fully working scraper which rolled along fine first time in Python. Was gonna close the chat, thinking I'll have to make some sort of executable, cos no way friend is gonna use command line. Then asked Claude: "Don't suppose you can make something in javascript with a HTML UI, providing we include a CORS proxy?" He says he doesn't see why not, that it would be quite an elegant solution and easy for someone to run. We discuss a few limitations this way, if scraped site had infinite scrolling etc. But agree it's a decent enough trade-off. I ask for some extra stuff in the HTML, including changing a very specific list of hardcoded URLs to be scraped to something that won't require me to update it for new sites or site changes. Just added a field to UI where user pastes snippet of code where what he needs scraped dwells. :=) Couple more minutes and Claude delivers something my friend will be able to use, just by opening a webpage. Looks nice enough, tells him exactly what to paste where and deliver outputs which show what he scraped that's new, updates his existing lists, orders alphabetically in CSV and counts/displays totals. As suspected, it doesn't work without CORS proxy, so he has a field to enter that too. Instead of being something designed for his specific needs, he can now use it to scrape pretty much anything. If he can't find the shiny code arrow that says "Scrape here" in any source code, he can just ask and I can tell him without having to touch code again. Has some limitations that python version doesn't. But pretty good, and will do more than my friend asked for, without me having to do anything but request a few things. Another thumb trending up for Claude, I guess.