Tip of the day: dump a website from command line

Here is another trick I daily use:
if you want to dump a website in a .txt file, you can use lynx. I added this alias to my ~/.bashrc

alias dumpwebtxt='lynx -dump -nolist -notitle'

and you will use it the following way:

dumpwebtxt "http://life-is-a-hack.blogspot.com/ > out.txt

Tip of the day: decrease MTU for a better wireless transmission

That's a small tip: when you are dealing with a noisy channel, you can decrease the MTU (1500 by default). Indeed, as an insight, if the noise destroys a "big" packet, there will be more to retransmit than if the packet was small. The drawback is that you will put more load on your devices, and sometimes some host will refuse the connection.

Anyway:

ifconfig wlan0 mtu 1000

where wlan0 is your wireless interface.

Note that one should not confound the MTU with the fragmentation threshold. The fragmentation threshold is one layer below the MTU.

Bittorrent: Ghost leeching is not dead !

I'm sure that lot of you user the bittorrent protocol to download movies & music. You know, when you are on these trackers, you have to maintain a certain ratio, usually .75 or 1. However, this is almost impossible to have this ratio, unless you've a 100Mbit symmetric Internet connection.

Typical solutions are to "cheat", by modifying the request send to the tracker. This is extremely easy to do, so I won't dig in that. However, cheating is BAD, and trackers have mechanisms to detect cheaters. At that time, there was "Oink" which was well known for its "cheater paranoia", and had a very good mechanism to detect them. Well, I tried 3x to cheat on this tracker and I got fired right away.

When you are in a tracking system, there is a very simple equation that must be fulfilled, this is:

outgoing packets == incoming packets

In other words, if someone uploads something, some other client will download this same file and vice-versa. If there is more uploaded data than downloaded, it definitely means that someone is cheating. Actually, the tracker cannot be that rigorous because the announcement are not done at the same time, so it probably uses different heuristics to determine whether a user is a cheater or not.

Here comes the plan: let's officially download a file from the tracker. Once you are getting it, you save all the seeders that are offering you the file (do a netstat for instance). So now, you have a list of IP addresses. Now, you connect on an another machine (with a different IP), and you start downloading the same file, but this time, without contacting the tracker. Basically, you can just alter the torrent file. After that, you inject your peers previously saved to your bittorrent client and TADaaa, you start downloading.

So what will happen next ? The seeders (uploaders), will report to the tracker they have uploaded X bytes. It will work once, twice, but after a while, the tracker will see that the seeder is uploading too much data compared to the people being in the network and will eventually fire him.

For the experiment, I modified the ctorrent client to support peer injection (PI) and disable the communication to the tracker when using PI. The client downloads indefinitely when it is in PI mode, so the seeder will report a high amount of uploaded data.


Be careful not to use the PI mode on your own machine, because this is extremely easy to detect this kind of download.

IPhone: Man in the middle attack very easy

I got a the new IPhone 3GS. Nothing to say, it's a very nice piece of software ! I cannot see the time the dev-team will release the jailbreak to have a real computer in the pocket :).
I was just trying to do a man in the middle attack on my iPhone to see what happened. I used ettercap, and did an ARP poisoning attack. Here is the result:

Huh, no way, I cannot see the certificate 0o ! ok let's believe it and click "Accept"... and you simply get the password :). Firefox is more agressive when something goes wrong (ie bad certificate), but here, the end-user probably doesn't what to do and simply click "Accept".

Poor man's wireless

I am not cheap, I am just a student. You know, here in Switzerland, Internet costs a lot: you've to pay your "link" (phone or TV - 20$/month), then the connexion is damn expensive for a single guy (50$/month). Moreover, it is well known that when you are a student, you have no money.

Thus, I decided to borrow my neigbourgs wireless connexion. Ok, I agree, nothing new here. We will say for the purpose of this article that an open connexion means a connexion that is "shareable" and we will only be interested in them.

When I got my first laptop, I was always scanning around trying to find wireless. It was not very conclusive, or sometimes I got such a poor quality signal that it was impossible to connect. Piano piano, I was looking toward a solution to increase this signal strength. When you speak about signal gain, you cannot think about not using an antenna. Good, that was my starting point: getting an antenna. However, few questions arised:

• Which antenna ?
• How to connect it with the computer ?
• How much does it cost ?

At that time, wireless hardware was more expensive that today, and difficult to find a dealer that accepts to ship to Switzerland... Anyway, I heard that an omnidirectional antenna is not that bad. I knew what an omnidirectional antenna was, well, at least more or less. Usually this is the kind of antenna you can see everywhere.

Omnidirectional antenna

At that time, I found a wifi-card on eBay: Senao 802.11b, 200mW with a mmcx connector. Actually, I was looking for a powerful card (I had the idea "the more powerful the best"), but also for a card accepting injection, and this criterion narrowed down the panel of wireless cards to 3 or 4 cards. This one was right in my budget !

My first PCMCIA card

When I received it, I was so happy to discover there were tens of network in my neigbourhood ! Moreover, lot of them were open :) (which is no more the case toay :( ). I was happy with this solution, with my Senao pcmcia card and my omnidirectional antenna. Obviously I was happy for few days, but after a while, I wanted more: how can I have a bigger signal strength, less noise, higher bitrate and a reliable connexion ? After studying all types of antennas, I figured out that a parabolic antenna was the top choice for my use (we will see why maybe later on). I finally ordered a 20.5dbi parabolic dish somewhere in France. So now, I still had my pcmcia card, with a big coax cable going on the parabolic antenna.

When I bought the Senao, I did not really take into account the fact that the card was only 802.11b compatible: how can you connect to a network that accepts only 802.11g ? Even worse, how can you sniff/jam/inject/replay (whatever) packets if you are not able to intercept them ? Raaaa, it was a mistake ! I decided to sell my Senao to buy another one, supporting 802.11g. I wanted a card fulfilling the following criterion:

• Atheros based chipset
• Powerfull
• External antenna connectors

Once again, it was not very easy to find. I finally bought the PCMCIA ubiquity SMC (SuperRangeExpress) 300mW, 802.11a/b/g with an MMCX connector. This card has pretty good performances, despite the fact you cannot use it without an external antenna. I could really feel the difference between the two cards, both in term of speed and reliability.

Now that I was fully armed, with my parabolic dish, I was able to connect to any network in range.

It was great, until I started to live with my girlfriend: she also wanted an internet access with her laptop and I definitely still didn't want to pay for internet when you can get it "for free". Now, I was facing a new problem: how can I provide internet to my girlfriend, too ?

• Connect my laptop to the antenna, as usual, and setting my laptop as an access point. This might work, but what if my computer is turned off ?
• Finding a was to efficiently repeat my neigborh's signal without the need of a computer in between. Huh, actually this device already exists and is called a "repeater".
  

I couldn't afford to buy a repeater, there were damn expensive, actually they still are. After some hours of google search, I figured out that a firmware called dd-wrt had a sort of "repeater" mode. Actually, it was totally unstable, but this was exactly what I was looking for. Moreover, at my parent's house I already had a linksys wrt54g, so I immediately flashed it. It was working pretty well, with a nice graphical interface. I decided to buy another one for my girlfriend and I. That was it, we had this linksys router on my balcony, attached to the parabolic dish and it was working damn well :). In a nutshell, we are repeating an unencrypted signal and set a new encrypted virtual network -> that's a bit paradoxical.
Then, I figured out that the buffalo whr-hp-g54 was more powerful (it got an amplifier), nicer, smaller and about half the price of the linksys one (~50$).

Buffalo whr-hp-g54

This was a pretty good choice actually and I am still using it. It has been resisting weather constraints for more than 3 years without any failure, it's just impressive. However, I confess that sometimes it's not working totally properly: for some reasons the bitrate is extremely slow.

So why "poor man's wireless" ? That's simple: router + antenna + connectors = 100$ which are around 1month and a half of internet.

Now that we have all the material, the fun can begin !

ability to solve CAPTCHAS == Threat ?

That's interesting to see how many web sites use CAPTCHAS nowadays. This reverse Turing test has been used for years now, to differentiate between a human and a machine, to avoid the problem of "spam accounts". To me, they are becoming more and more unreadable (cf google captcha), because they are trying to increase the security by reducing the segmentation, rotating/difforming the letters, etc.

Google captcha

Do the end-user really care about these captchas ? I'm not really sure... However, I can see multiple ways to take advantages from them. Imagine for example you want to make your website more known on Stumbleupon or Digg. An easy solution would be to create multiple accounts and vote for your site ("I like it") in order you to be in the top ranking. By using a web bot - like the one explained in the previous article for the SMS - makes the job extremely easy to achieve.

But, the only point is that we'd like to automatize the job of creating new accounts and therefore, solving captchas.

StumbleUpon captchas

Current methods claim they can solve them with a probability of 30% (for Google's one), using pattern matching techniques like support vector machines. That's not bad, but I definitely think that there is room for improvement. Few techniques to solve them have be explained here. On the other hand, websites proposing captchas let you try few times, so it's not a big deal if you have to let your bot run 3 times longer to get the desired result.

Well, I'm really interested to dig into this problem as soon as I'll have a bit more time to fuck around :)

Small hack to send free sms, totally in bash...

Let's start with my first real post. I'll present you the power of curl. As you probably know, curl is "A Client that groks URLs". For this example, I will use a website that provides SMS service for free.

I was kind of pissed of to have to login every time I wanted to send an SMS/MMS... Well, it takes around 40-50 seconds to enter the login, password, then write the message, enter the phone number of the friend and ship it... If you do it, let's say 10 times daily, it's around 10 minutes that are completely lost. Thus, I decided to a shell script that automatizes this, and launch the script from my shell.

First, let's see how it works:

$ sms sab "Hi :)"
Remaining SMS: 481

$ sms 0794046789 "Hi man, I'm sending you an SMS from my box :)"
Remaining SMS: 480

As you can see, it acceps both names & numbers.


First, in a file called contact.txt I write all the contacts I wish to have the number, the name and the number are separated by a TAB :

$ cat contact.txt
toufic 0793023393
eric 0783034336
sab 0773346337
guigui 0768228392

Then, it becomes extremely easy to get the number if it exists in this file:

NUMBER=`cat $CONTACT |  grep $1 | awk -F"\t" '{print $2}'`
if [ ! -n "$NUMBER" ]
then
 NUMBER=$1
fi

Now that we have the number, we can do some processing on the number and the message

PREFIX=`echo ${NUMBER:0:3}`
SURNUMBER=`echo ${NUMBER:3:10}`
NUMBER_LENGTH=`echo $NUMBER | wc -m`
MESS_LEN=`echo $MESSAGE | wc -m`
MAX_LENGTH=600
if [ "$MESS_LEN" -gt "$MAX_LENGTH" ]
then
 echo "MESSAGE TOO LONG (Max 600 char). Exiting..."
 exit 1
fi

if [ "$NUMBER_LENGTH" -ne "11" ]
then
 echo "Bad Number ($NUMBER_LENGTH)"
 exit 1
fi

Here we want a message being smaller than 600 chars and a correct number, ie with 11 digits. Note that the preffix is 3 digits long. Now, we want to set up the connexion with the server (ie to login) and get the cookies:

# Initial connexion. The cookie is saved in $COOKIE
$COMMAND $SITE/$PAGE -D $COOKIE > /dev/null
# Now ship the message with the correct number
$COMMAND -e $SITE/$PAGE -A "Opera/9.23" -D $COOKIE2 -b $COOKIE \
-d  "isiwebuserid=$LOGIN&isiwebpasswd=$PASS&isiwebjavascript=No&isiwebappid=mobile&isiwebmethod=authenticate&isiweburi=%2Fyouth%2Fsms_senden-fr.aspx&isiwebargs=login&login.x=0&login.y=0" \
$SITE/$PAGE_AUTH > /dev/null

Ok, I agree, it sucks here. I retrieved this lonnnnng URL by using Paros . So basically, we do:

• -e: Sets the Referer as being the official page (ie $SITE/$PAGE).
• -A: the User-agent, here Opera.
  
• -D : Saves the new cookie.
  
• -b : Use the previously saved cookie
• -d : Send a POST request to "$SITE/$PAGE_AUTH". You can see that the request has $LOGIN and $PASS
  

Now we are logged in and we got the cookie for the session. We are now able to send the SMS:

$COMMAND -e $SITE/$PAGE -A "Opera/9.23" -b $COOKIE2 \
-d "__EVENTTARGET=&__EVENTARGUMENT=&__VIEWSTATE_SCM=1&__VIEWSTATE=&CobYouthSMSSenden%3AtxtMessage=$MESSAGE&CobYouthSMSSenden%3AtxtNewReceiver=$NUMBER&CobYouthSMSSenden%3AbtnSend=Envoyer&FooterControl%3AhidNavigationName=Envoi+de+SMS&FooterControl%3AhidMailToFriendUrl=yoblabla.aspx" $SITE/$PAGE_SMS 

As you can see, $MESSAGE and $NUMBER have been replaced in the request. Again, I found this url by using Paros.

That's it... we are now able to send sms from the command line for free, without having to log in every time. Note that we can take back the result from curl and do some parsing on it. This is how I get back the "Remaining SMS". A cool application is for example when you are monitoring a special activity on your network and you would like to be informed when something strange is happening... just call the sms script and it will inform you.

Typically, I used this kind of technique to set up automatically accounts and vote to get invitations to a concert.

First post... Hello

Hi there... first post on my first blog... cool, sounds easy. I'll try to make small articles on every days "hacks" that simplify my life.
Let's see how things go :)