The Quick and dirty:

New PasteLert lives at http://andrewmohawk.com/pasteLertV2/

Downloads:

» Interface -> http://andrewmohawk.com/pasteLertV2/src/pastelertv2_Interface.zip
» Cron Tasks -> http://andrewmohawk.com/pasteLertV2/src/pastelertv2_Cron_Tasks.zip
» Scraping Script -> http://andrewmohawk.com/pasteLertV2/src/pastelertv2_Python_Scraping_Script.zip

And of course if you want everything -> http://andrewmohawk.com/pasteLertV2/src/pastelertv2_all.zip

Overview

My linode has been pretty much falling over due to the previous version of the pastebin alerts for a number of reasons:

» Scripts sometimes get blackholed (pastebin.com allows the connection but doesnt respond – due to their DDoS protection)
» Scripts sometimes were still running when the PREVIOUS script had not completed causing a chain reaction of fail
» Deletes would be happening while the above scripts where running causing MySQL to tilt

Lucene/Solr

As such I recently re-worked the service. Initially I started playing around with other DB types to try and get my Linode to store more than a day or 2s worth of Pastebin.com data. I looked around and it appeared that Lucene/SolR was the solution I was looking for, and actually it does work _very_ well at storing large amounts of data (I had it running with about 2 weeks of data). However there were a number of issues:

» After about a week or 2s worth of data (avg around 20-30K posts a day, x 14 = 280 000 – 420 000 posts) the search times were SLOW (talking something like 5-15 SECONDS)
» Because Lucene is not a RDBMS there is no concept of having something like a row ID or an auto-incrementing ID – so this would have to be handled by the script to get the number of entries and +1 every time
» Because of the above Alerts would have to work on a date (when the post was made – so working out from x secs ago or y minutes ago), and an ISO formatted date no less (no unixtime) it became a real pain.

However, with that being said I did still build the interfaces for it and if you are looking to implement it with SolR / Lucene just message me for the schema and Python/PHP scripts.

Basics

Ultimately however I decided to stick to the same system previously used but rather than have cron’d scripts that pull the data have one long running python script that you can place in the background. Pretty basic and the code should be self explanatory, the gist of it:

1. Pull archive.php from pastebin.com [ http://pastebin.com/archive.php ]
2. Extract all the paste entries with a regular expression ( re.compile(‘<td><img src=”/i/t.gif” .*?<a href=”/(.*?)”>(.*?)</a></td>.*?<td>(.*?)</td>’,re.S) )
3. Check if we have seen it in the last 500 or so (that we have in a python list), if not, pull the raw paste
4. INSERT IGNORE (in case we missed a double) this data

Then for the “alerts” themselves, basically:

»Every 30 minutes (or whenever you set the cron to run) search if the terms in the database have been seen
»If seen send out mail

Additionally of course there is a web interface that you can use to add alerts as well as search the current index’d pastes.

Downloads / Config

My Crontab at this stage looks as follows (if you want to just copy mine):
*/20 * * * * php /home/andrew/pasteLertV2/Cron_Tasks/sendAlerts.php
0 1 * * * php /home/andrew/pasteLertV2/Cron_Tasks/truncPastes.php

And i’ve kicked off the script that puts the data in the database with:

andrew@mothership:~/pasteLertV2/Python_Scraping_Script$ nohup python scrapePastebinMySQL.py &

I’ve seperated the scripts into the 3 sections:

» Interface -> http://andrewmohawk.com/pasteLertV2/src/pastelertv2_Interface.zip
» Cron Tasks -> http://andrewmohawk.com/pasteLertV2/src/pastelertv2_Cron_Tasks.zip
» Scraping Script -> http://andrewmohawk.com/pasteLertV2/src/pastelertv2_Python_Scraping_Script.zip

And of course if you want everything -> http://andrewmohawk.com/pasteLertV2/src/pastelertv2_all.zip

Essentially the only modification you need to do is within the interface / cron tasks modify the ‘setDB.php’ script with your db credentials and within the scraping script, set these on line 141.

-AM

<responsible_disclosure>
Before i discuss this, let me just say that the bug has been patched (was in 2.5.1) and at the time of writing this Joomla is already 2 increments away -  2.5.3 is currently available.
</responsible_disclosure>

Overview

So back in the dark ages of my programming life I, like many people who started coding, worked in web development. And during these times I had to write modules/hack things together for various frameworks, including Joomla. At that stage i was also signed up to the Joomla security security list and a few weeks ago i saw a security update come through that affected most joomla installs and was a core issue. Most of the ‘omgjoomlasux’ commercials/vulns/notifcations actually are problems with 3rd party modules rather than with joomla itself, so this was pretty interesting. Secondly the bug was listed as a SQL injection bug and critical, this gave me the idea that with a bit of luck and some mysql commands I too could hack the planet. I fired up my green_text_on_black_background console and gave it a whirl.

First i pulled Joomla installs 2.5.1 and 2.5.2 from the download page (the issue had just been patched), next I had to go through these to figure out what changed. Doing a quick diff in linux, or for windows people use the cool winmerge it was quick and painless to find the issue:

./plugins/system/redirect/redirect.php:

Joomla251:
$db->setQuery(‘select id from ‘.$db->quoteName(‘#__redirect_links’).” where old_url=’”.$current.”‘”);

Joomla252:
$db->setQuery(‘select id from ‘.$db->quoteName(‘#__redirect_links’).” where old_url=’” . $db->quote($current) . “‘”);

So right off the mark, things are looking great, got a SQL command that is not escaped via the $current variable. A quick search to find this in that function gives us:

$current = $uri->toString(array(‘scheme’, ‘host’, ‘port’, ‘path’, ‘query’, ‘fragment’));

Excellent, something we can regularly manipulate with just a browser (since its adding the URI from the browser), so now what could I do with this information?

First off i knew what the SQL query looked like, so it was a lot easier to manipulate in a SQL interface or even something like PhpMyAdmin. So modifying the query quickly gave me some failboats:

1. I couldnt do any insert / modifcation of data from a secondary query like:
- UNION SELECT (insert x into y)
This was due to the fact that insert x into y simple didn’t return anything so it could not be joined to the previous SQL query, the Union requires a returned value to join to the current SQL statement. I tried doing things like SELECTing an INSERT, using the IF statement and a few others without luck.

2. MySQL doesn’t have anything like xp_cmdshell so I could not off-the-bat execute raw code (sadface).

However, MySQL did have a few functions that were super useful:
* INTO DUMPFILE – this lets me write files out to the system (winning.) Unfortunately I have NO IDEA where the webroot is, and in the testing I did MySQL almost never had write access to the webroot when i knew where it was
* SELECT LOAD_FILE – lets me select local files into things – Great apart from the fact that I cannot write into the database and I dont know where I can put files

At this stage it was super-facepalm-time. But then with a little help from Roelof and the internets i started looking at timing attacks.

Read more »

Overview

I’ve always been semi interested in botnets/trojans and targetted attacks and the way they get their data in and out and how the command and control centres work. One of the things i’d usually do is see if I can determine where the traffic is going from the bot (infected machine) and this would obviously point me to the c&c. I’d then fire up Maltego and start playing with that IP/hostname to see where else it appears, what other things are linked to it and so on. One of the concepts I was playing around with was how could you hide where your c&c and from this FireBridges as a concept where created.

Since we were playing badguy-badguy I decided to think how do the good guys go about taking apart a bot to get to your c&c and i figure it probably works something like this:

* What is c&c.thisisnotnormal.traffic.com – browse to it, portscan, etc
* Look at traffic going to c&c.thisisnotnormal.traffic.com – replay it to see results
* Take apart the traffic and start sending modifying parts to see results
* Go and literally pick up the machine(s) hosting c&c.thisisnotnormal.traffic.com

So how would you go about making these peoples lives a little more painful?

* Make sure no connections go directly to the c&c – route through proxies
* Make sure all traffic is encrypted/encoded and if either fails destroy the proxy
* All proxies look for replay attacks and destroy themselves after a threshhold (could be 1 for the super paranoid)

Basics

From this the idea of Firebridges (really thought it was a cool name but i see there are loads of other things with the same name) were born. The idea is relatively basic:

* You have a series of proxies that dont know about anything apart from the nextHop in the chain
* Proxies all make sure that data passing through is correctly encrypted (checking for tampering)
* Proxies all make sure data is not being replayed
* If a proxy detects something going wrong it removes all files associated with the nextHop leaving the people chasing you with a dead end

Implementation was not too difficult, whipped something up in PHP that works like this:

* All requests to nextHop include a POST variable ‘key’ that contains a key made up of the following (B64(RIJNDAEL256(B64(secretkey))):

1. b64_1 = Base64_encode(‘text’)
2. RIJ_2 = RIJNDAEL_256_encode(b64_1)
3. b64_3 = Base64_encode(RIJ_2)

* All requests hit a ‘bridge.php’ page that does:
* @Include ‘proxy.php’, call function proxyRequest(); which checks auth above and replay attacks via SQLite db
* If proxyRequest() returns false, remove the SQLite database and ‘proxy.php’ script leaving the person chasing you with a 5 line php file that once included something
* If proxyRequest() returns != false, simply return the page to the browser.

Results

Using FireBridges, you can now create a proxy network easily by simply changing the nextHop variable in proxyRequest.php and adding them to machines all over the world that will burn if anyone tampers with them. This means if anyone is investigating why traffic is going to thisisauniquehostname.weareevil.com and decides to browse to it the proxies will burn themselves (and they will get the default apache page – configurable in bridge.php) and by the time they pick up the machine all they will have is 1 php script that gives away nothing. It also means that if these investigators are slightly more resourceful in their approach and try replay the attack after a certain threshold of replays (default is 2) the next replay will burn the proxy. Finally if they are even more resourceful and try tamper with any of the data the proxy will burn on the first attempt presuming it doesn’t match your requirements.

Read more »

So this is going to be a rather strange post as at the time of writing its not actually implemented, the system is built in a waterproof container as well as the networking setup and so on. I figure that since I will only be able to get another Arduino and ethernet shield at a later stage I may as well write it up for now. Below are a few pictures of the system completed:

 

 

 

 

 

 

 

 

 

With regards to the requirements for the system my part spec was as follows:

• One large reservoir – I got an 80 litre orange bucket for about R100
• Arduino + Ethernet shield – pretty stock standard
• 4x 10K resistors – used for the sensors
• 4x ‘sensors’ – sensors setup as before, coiled wire (soldered if you can) and taped on
• 8x galvanised steel washers – used as the actual sensors
• 2x transistors - used for the relay setup
• 2x relays - I used LT-5GS’ for this to switch the pumps on and off
• 2x Diodes – used for my relay setup
• 2x Water pumps - I used two (1 per pot) honestly because it was cheaper, although not as elegant as having a electrical valves and a more intricate watering system, mine were the 1.5A 12V bilge pumps (about R150 each)
• 2x Water pump power supplies – Obviously used for the pumps power, I used some cheap power adapters that didn’t cost much
• 1x Arduino power supply - See http://www.arduino.cc/playground/Learning/WhatAdapter for more information
• Wires, Tape, Tv Series, Patience -  essential in setting this up :)

Read more »

I’ve always been harping on about growing my own tomatoes and other veggies and earlier this year i attempted it for a while.Unfortunately with me going away for various conferences and generally being a forgetful lout i managed to kill many many plants!

What i wanted was:

What I got was:

So recently I was playing with my arduino and thinking about this, and got the idea to try and create an automated gardening system where my plants where automatically given water/light/etc without me having to worry about it. There are some fantastic resources online like http://www.instructables.com/id/Garduino-Gardening-Arduino/ and http://makeprojects.com/Project/Garduino-Geek-Gardening/62/1.

I began planning something i’d want, and ideally it would have to be this:

• Moisture control for water
• Water pump to water them
• Light sensors for Lights and LEDs (red and blue for optimal growth)
• Humidity to keep my plants cosy
• Interface via LCD/Web to see how things are doing (if more water is needed etc)
• Solar panel to allow the system to be completely stand alone

Read more »

Sorry guys, I noticed that I haven’t been getting any pasteLert updates, and i just realised why (see above picture for my reaction).

Change line 4 in truncPastes.php from:

mysql_query(“truncate pastebin”);

To:

mysql_query(“delete from pastebin”);

 

Explanation:

Truncate automatically resets the auto-incrementing IDs so that when the table was truncated pastes started from ID 0 again, which when checked against what the last ID sent to the user was obviously lower. Sorry for the headache, to fix it, apply the above then run:

update alerts set LastID = 0;

 

Mah bad,
-AM

Ohhi

Archive for pasteLert

So i finally got round to putting the source together and writing this out. We’ve been really busy with Blackhat training at work and so on and i’m generally just lazy. Also releasing now mostly because the mysql database on my linode keeps crashing, its just too small a box to keep *all* pastebin entries. The code is messy, so expect arb/no commenting but its pretty straight forward, feel free to shoot through any questions you have. Also i messaged pastebin to see if they’d implement something like this or let me do it, but i didnt get any responses to any of the messages :(

Anyway, here is the basic rundown:

• Setup your mysql, create a database ‘pastebin’ – Google will give you this info :D
• Drop the structure in, its in the archive as pastebinStructure.sql. mysql -u root -p pastebin < pastebinStructure.sql
• Extract the archive to its own directory in your webroot, preferably ‘pasteLert’ :)
• Change the setdb.php file to your mysql details. Edit alerts.php to include your email and location information rather than mine
• Setup the crons as below

Crons:

Basically there are 4 cron jobs that you need to add:

• pullPastebin.php – this will go to http://www.pastebin.com/archive.php and get the pasteIDs and add them to `pastebin`.`pastebin`, I generally run this every 2 minutes and my cron looks like this:
  • */2 * * * * php /var/www/html/andrewmohawk.com/pasteLert/pullPastebin.php
• pullPastes.php – this script then goes and pulls each paste with a random delay between 0-5 seconds (see line 14 if you want to change that). I generally let this run every 10 minutes and looks as follows:
  • */10 * * * * php /var/www/html/andrewmohawk.com/pasteLert/pullPastes.php
• sendAlerts.php – this script sends out the alerts via email, this is really up to you, obviously as close to 10 minutes means its as close to when you have the data, mines at 15 mins:
  • */15 * * * * php /var/www/html/andrewmohawk.com/pasteLert/sendAlerts.php

Cron Part 2!
So the reason my box was falling over was that every day i’d push all the pastebin’s from that day into another table (pastebinOldData). Essentially i have now changed mine to stop doing this and rather truncated the daily log instead of saving the data.  You however hopefully have a bigger box and can store all the data, or you can always just truncate the data, so you need to pick one of the two files, either truncPastes.php or rotatePastes.php.

Truncate:
0 1 * * * php /var/www/html/andrewmohawk.com/pasteLert/truncPastes.php

Rotate:
0 1 * * * php /var/www/html/andrewmohawk.com/pasteLert/rotatePastes.php

I think that pretty much covers it, feel free to mail in what you are looking for if you need any help.

Kthnx,
Andrew

ANDREW I DONT CARE ABOUT YOUR STORIES! JUST GIVE ME THE LINK! >> http://andrewmohawk.com/pasteLert/

Hey guys,

So here is my latest project, extending from the previous pasteScraper to do something a little different with the pastebins. Essentially i recreated google alerts but with a bit more searchiness (yes, i make up words now too).

How it Works

• I enumerate all new pastes from http://www.pastebin.com/archive/ every minute and add them to a ‘download’ queue.
• New pastes are then downloaded to a local database
• Alerts are periodically cron’d
• Search functionality is via a fulltext search of pastes

What does it give me?

• The ability to search for *anything* on pastebin.com
• Semi-realtime searches
• Email alerts when your term is hit!
• RSS feeds for searches
• The ability to search with AND keywords in pastebins

How it is all going to fall apart

I dont really see this as a long term project, merely something that shows a PoC for how much stuff is leaking out via PasteBin.com and how cool it really is. Some issues i see that may happen with this:

• People will switch to more secure pastebins that don’t allow indexing, don’t have archive pages and arent indexed by search engines
• My small linode will fall to pieces because the fulltext like queries are painfull
• Pastebin.com will not be impressed with me doing this and start blocking it

Linkage

http://andrewmohawk.com/pasteLert/, feel free to play/comment/etc :)

-AM

 

p.s. Thanks to Chris Hadnagy and Roelof Temmingh :D

So this is the only entry i have where i’ve built an app that wont work from day 1.

“Oh why andrew, why?” you might say, but as I shrug, this was not my fault. So a while back Samy Kamkar produced his geolocation proof of concept code which works awesomely at being able to take an AP MAC Address to GPS Co-ordinates. Sadly now however Google location services seem to be broken at the moment with both Samy’s and my code being horribly broken with the results either being the same GPS Co-ordinates or 0,0.

This happened of course only *after* i had completed my app (had to do multithreading and slap a design on it) .

But anyway, here is the basic idea of it:

• Recieve in a NON GPSd airodump-ng csv file
• Parse out all of the AP MAC Addresses
• Take these to GPS co-ordinates to street addresses via opennominatim
• Return this response either as a KML file (plotted on google earth)
• OR as a text file to simply display the results

So anyway, you can have a look at the application over at http://andrewmohawk.com/airodumpvsgeo/ . Currently still in my shite naming ‘convention’ this one is called “AirodumpvsGEO”.

-AM

So i was chatting to Chris Hadnagy and he was having a bit of an issue getting an API key for yahoo BOSS and it seemed troublesome.

So i popped off a mail to yahoo to ask how i could get a key to share and they basically said i can just use mine. So i quickly repackaged the pasteScraper with my key so now its as simple as extracting the zip in your webroot directory and browsing to it :)

Get the new one here

Of course you can still use the one on my site over at http://www.andrewmohawk.com/pasteScrape/

Enjoy!
-AM