The Quick and dirty:
New PasteLert lives at http://andrewmohawk.com/pasteLertV2/
» Interface -> http://andrewmohawk.com/pasteLertV2/src/pastelertv2_Interface.zip
» Cron Tasks -> http://andrewmohawk.com/pasteLertV2/src/pastelertv2_Cron_Tasks.zip
» Scraping Script -> http://andrewmohawk.com/pasteLertV2/src/pastelertv2_Python_Scraping_Script.zip
And of course if you want everything -> http://andrewmohawk.com/pasteLertV2/src/pastelertv2_all.zip
My linode has been pretty much falling over due to the previous version of the pastebin alerts for a number of reasons:
» Scripts sometimes get blackholed (pastebin.com allows the connection but doesnt respond – due to their DDoS protection)
» Scripts sometimes were still running when the PREVIOUS script had not completed causing a chain reaction of fail
» Deletes would be happening while the above scripts where running causing MySQL to tilt
I saw the pastebin guys put out a list of the IP addresses that have been attacking them for people to check if they were, I wrote a quick little script to test this at: http://andrewmohawk.com/pastebinAttack/
Secondly, i see pastelert broke with the new GUI change on pastebin, I’ve fixed it on mine and I will post an update sometime here, if its urgent just drop me a mail and i’ll send the patch :)
Sorry guys, I noticed that I haven’t been getting any pasteLert updates, and i just realised why (see above picture for my reaction).
Change line 4 in truncPastes.php from:
mysql_query(“delete from pastebin”);
Truncate automatically resets the auto-incrementing IDs so that when the table was truncated pastes started from ID 0 again, which when checked against what the last ID sent to the user was obviously lower. Sorry for the headache, to fix it, apply the above then run:
update alerts set LastID = 0;
So i finally got round to putting the source together and writing this out. We’ve been really busy with Blackhat training at work and so on and i’m generally just lazy. Also releasing now mostly because the mysql database on my linode keeps crashing, its just too small a box to keep *all* pastebin entries. The code is messy, so expect arb/no commenting but its pretty straight forward, feel free to shoot through any questions you have. Also i messaged pastebin to see if they’d implement something like this or let me do it, but i didnt get any responses to any of the messages :(
Anyway, here is the basic rundown:
- Setup your mysql, create a database ‘pastebin’ – Google will give you this info :D
- Drop the structure in, its in the archive as pastebinStructure.sql. mysql -u root -p pastebin < pastebinStructure.sql
- Extract the archive to its own directory in your webroot, preferably ‘pasteLert’ :)
- Change the setdb.php file to your mysql details. Edit alerts.php to include your email and location information rather than mine
- Setup the crons as below
Basically there are 4 cron jobs that you need to add:
- pullPastebin.php – this will go to http://www.pastebin.com/archive.php and get the pasteIDs and add them to `pastebin`.`pastebin`, I generally run this every 2 minutes and my cron looks like this:
- */2 * * * * php /var/www/html/andrewmohawk.com/pasteLert/pullPastebin.php
- pullPastes.php – this script then goes and pulls each paste with a random delay between 0-5 seconds (see line 14 if you want to change that). I generally let this run every 10 minutes and looks as follows:
- */10 * * * * php /var/www/html/andrewmohawk.com/pasteLert/pullPastes.php
- sendAlerts.php – this script sends out the alerts via email, this is really up to you, obviously as close to 10 minutes means its as close to when you have the data, mines at 15 mins:
- */15 * * * * php /var/www/html/andrewmohawk.com/pasteLert/sendAlerts.php
Cron Part 2!
So the reason my box was falling over was that every day i’d push all the pastebin’s from that day into another table (pastebinOldData). Essentially i have now changed mine to stop doing this and rather truncated the daily log instead of saving the data. You however hopefully have a bigger box and can store all the data, or you can always just truncate the data, so you need to pick one of the two files, either truncPastes.php or rotatePastes.php.
0 1 * * * php /var/www/html/andrewmohawk.com/pasteLert/truncPastes.php
0 1 * * * php /var/www/html/andrewmohawk.com/pasteLert/rotatePastes.php
I think that pretty much covers it, feel free to mail in what you are looking for if you need any help.
ANDREW I DONT CARE ABOUT YOUR STORIES! JUST GIVE ME THE LINK! >> http://andrewmohawk.com/pasteLert/
So here is my latest project, extending from the previous pasteScraper to do something a little different with the pastebins. Essentially i recreated google alerts but with a bit more searchiness (yes, i make up words now too).
How it Works
- I enumerate all new pastes from http://www.pastebin.com/archive/ every minute and add them to a ‘download’ queue.
- New pastes are then downloaded to a local database
- Alerts are periodically cron’d
- Search functionality is via a fulltext search of pastes
What does it give me?
- The ability to search for *anything* on pastebin.com
- Semi-realtime searches
- Email alerts when your term is hit!
- RSS feeds for searches
- The ability to search with AND keywords in pastebins
How it is all going to fall apart
I dont really see this as a long term project, merely something that shows a PoC for how much stuff is leaking out via PasteBin.com and how cool it really is. Some issues i see that may happen with this:
- People will switch to more secure pastebins that don’t allow indexing, don’t have archive pages and arent indexed by search engines
- My small linode will fall to pieces because the fulltext like queries are painfull
- Pastebin.com will not be impressed with me doing this and start blocking it
http://andrewmohawk.com/pasteLert/, feel free to play/comment/etc :)
p.s. Thanks to Chris Hadnagy and Roelof Temmingh :D
So i was chatting to Chris Hadnagy and he was having a bit of an issue getting an API key for yahoo BOSS and it seemed troublesome.
So i popped off a mail to yahoo to ask how i could get a key to share and they basically said i can just use mine. So i quickly repackaged the pasteScraper with my key so now its as simple as extracting the zip in your webroot directory and browsing to it :)
Of course you can still use the one on my site over at http://www.andrewmohawk.com/pasteScrape/
Yeah im really lazy, so im not gonna write a lot about it, basically, if you wanna use it on my site hit it up at http://www.andrewmohawk.com/pasteScrape/ otherwise feel free to download it and run it yourself from this archive
Basically, go to http://www.andrewmohawk.com/pasteScrape/ and try some of these:
- “gmail/facebook Password” – free facebook/gmail/whatever accounts
- “rbot” – find rbot config files, including the passwords and the irc network that it connects to, ie, if you have an irc client you get a free botnet
- “enable password” – Cisco goodness
- “BEGIN PGP” – pgp keys anyone?
- “DB_PASSWORD” – loads of database passwords
- “Shellcode” or “Exploit”
You get the idea :)
Enjoy my readme after the break :D
1 // What is PasteBinScraper?
2 // How does it work?
3 // How do i use it?
4 // How do i install?
5 // Extending
6 // TODO
7 // Thanks
Read more »
- ZaCon V Badge [2/2]: How they work
- ZaCon V Badge [1/2]: Build Time
- ZaCon V: Badge Sneak Peak *update*
- Kingphisher: Semi-automated phishing
- Magnetic Stripes: Part 2 (Attacking)
Not the quickest of cats
on the best of days.
Magnetic Stripes (2)
- April 2014
- January 2014
- November 2013
- October 2013
- March 2013
- January 2013
- December 2012
- September 2012
- July 2012
- May 2012
- April 2012
- February 2012
- November 2011
- October 2011
- September 2011
- August 2011
- June 2011
- April 2011
- March 2011
- February 2011
- January 2011
- November 2010
- October 2010
- September 2010
- March 2010
Created by Site5 WordPress Themes.
Experts in WordPress Hosting.