14 November 2018

phpVatch: Websites Leaking Unsafe PHP Versions

As practitioners of IT whether be it infrastructure , web development or information security we would have had the interaction with PHP and related requirements in various stages of our professional experience.

That being said, what is so critical about this End of Life (EoL) is that , beyond 31 Dec 2018 both versions including 5.6 and 7 would be cut off with critical patch updates. In other words, even if vulnerabilities are identified & highlighted, there won't be patches released for these two versions as per PHP supported versions announcement.

Though the original EoL for v5.6 was set at Jan 2017, it was extended until 31 Dec 2018 for obvious reasons of high demand and large scale implementation giving ample time and scope for migrations. In the end, they have decided to no longer extend the cut off date.

Considering the fact, even the vast majority of the websites still run on v5.6, this is going to be an uphill task for sure. As per ThreatPost , neither WordPress nor Joomla have provided a clear road map on the migration plans needed. Drupal has released an update that effective 6 Mar 2019, the minimum requirement of PHP would be versions 7.1+.

With the deadline nearing closer and closer, not much of action is seen from infrastructure side on the migrations to v7.2 or above. A quick scan of the Top 1 million websites offered by Alexa, clearly shows only 4.88 % is currently running with v7.2 or v7.3.

PHP Versions

With all these information available, what can we do to increase the adoption of v7.2 on the sole basis of security was to highlight to respective websites on the current risks through public channels and forums.

In this attempt, I tried to scan the entire Top 1 million of Alexa and found to my surprise , not only we have very low adoption,there were so many websites that are leaking their PHP versions in web servers header information itself. As a best practise and part of server hardening, version information should be turned off and that is missed as well giving ample scope to check their individual versions on websites leaking such data.

This PHP information leak can be easily fixed with a simple configuration which is clearly documented in PHP.net website itself.

With my scan data of the websites leaking PHP versions from  Top 1 million websites, parsed the data quickly and created a simple phpVatch which can showcase the findings in a simple form.

All the data points including the folks who have migrated successfully are presented in phpVatch Hall of Fame.

Interesting enough, this scan and the entire process of data collation and interpretation has given me a good experience and I wanted to share the same over here. Simple glance at the above graph clearly shows the alarmingly high and hovering wave of unsafe versions over the safe one which is pretty close to the border line.

Technology used :

Well, I fancy Python for all my scripting work ( sorry PHP fans ) and I tried to whip up a simple scanner to get the header information of the websites from the list.

Differences between multi-threading and multiprocessing made more sense and I ran my initial script in a multiprocessing mode and to my utter dismay it was able to scan very slowly. That clearly shows that I was not prepared well enough to do this and I went back to drawing board and started scouting for ways and means of increasing the efficiency of this script.

Voila. I found a solution which was reasonably good enough to meet my requirements without demanding too much of infrastructure computing power. It was the combination of RabbitMQ and Celery running my python scanner in a smooth manner which actually delighted me considering the initial failure. Flower was one beautiful  tool which I used for progress  monitoring. Within couple of hours, I was able to get all the data and now I was left with the task of presenting the same.

Took the regular route of Bootstrap with simple API to serve the contents for phpVatch and completed the same, similar to my earlier project XposedOrNot: Want 850 Million Passwords For Free?.

All scan data is sorted alphabetically and stored as plain text and the API feeds on this raw data for providing the web page with content. The idea of using API instead of static page is to provide the opportunity of refreshing the data at regular intervals and not touching the front end interface.

Couple of things I am happy about this project is that, though phpVatch caters to a very small target who might be interested in PHP, this has given me the opportunity to learn and effectively use more tools which can benefit all. For example, just ran the site through Google Chrome's Audit under the Developer tool set and to my excitement found that it scored 100% on all three parameters checked, which by itself is a great milestone for me.

Feel free to visit the site at https://xposedornot.com/phpvatch/.

For privacy conscious user, the same is also reachable at xposedwxxe7bbqlz.onion/phpvatch.

Would request all the readers to pass on this critical PHP EoL to folks who need to migrate. Let us all help each other in improving and making the web as a safe and secure one. 

Next steps :

As a next step, I would be updating this once every 10 days  to look at the positive improvement. 

Feel free to share in your feedback and comments down below for the benefit of all . 

Update 21-Nov-2018

Update 2-Dec-2018
Update 1-Jan-2019

28 June 2018

XposedOrNot: Want 850 Million Passwords For Free?

Glad to inform that today I have released ~850 Million passwords used in real life for all to check their password posture and understand their credentials exposed. This will help each and every one to check the data exposure related to their current, past or future passwords.

The primary idea of this tool is to provide feedback through a simple web based console accessible as https://XposedOrNot.com. The same is also available as an API to be consumed by any application or enterprise for strengthening their passwords posture. API and related information will be explained in few minutes.

How did I source these plain text passwords ?

Now comes the big question of how I am able to source such a huge amount of plain text passwords. Well the answer is pretty simple. There are various places through which we can source these passwords, provided you have the time and energy to find and sort them.

Finding them is easy, but sorting them from their raw sources is too painful and I had to agree that sorting takes more time compared to sourcing the passwords.

Let's start with the infamous 1.4 billion collection which was touted as the largest single collection of plain text found in dark web. Well, that's  one of my easiest and less painful one to extract the passwords. This dump gave me 1,400,553,869 ( 1.4 Billion +) passwords out of which 463,625,357 ( 463 Million ) were unique.

My next source of passwords were extracted from a large torrent of such dumps being shared across in breach exposure websites. That gave me another massive collection of  passwords. Not to mention, another infamous collection was from ExploitIN dump which contained 793,372,299.

One another interesting thing I have to mention here is that during this scouting process, I came into possession of enough data breaches and their raw exposed files spread across on plain Internet in various dump sites.

Couple of torrents sharing data breach dumps with sizes exceeding into 100s of GB in size. Since most of the breaches except baring few have their passwords stored in various forms of easily recoverable MD5 hashes to the not to easy bcrypt hashes, was wondering a lot about what to do with all those hashes.

I do not have the huge processing capacity needed to crack these passwords and at the same time not ok to crack those passwords myself. This is where hashes.org comes in handy, brilliant set of folks who have taken the time and effort to crack most of these exposed breaches.

My sincere thanks to them for their work has helped me a lot in this exercise. In fact, their collection was so good that they even had cracked passwords which were not even in my repository of breaches saved so far.

After crossing couple of hundred millions of unique passwords, the next obvious search led me to scrape pastebin for exposed credentials that are posted there very frequently and most of the times anonymously. This one took even more time and effort as it was not a direct one. First, we need to identify pastes which have passwords exposed.

Twitter and Pastebin sources

To identify that, I ran couple of twitter searches and identified accounts which more or less contained such information befitting the purpose. With a positive search output, started collecting all those tweets and that came around to 501,032 in number. Oh yes, its a large number of URL's to be downloaded from pastebin. Next is to download all the pastes and quickly got impacted by the restrictions enforced by pastebin on scraping.

Interestingly, at the same time pastebin gave me an option to download the entire stuff through a white listed IP address and it costed less than $15 for life time access called PastebinPro. Looked like a good deal and I picked it up. So I tasked a simple python script to download the entire 501K addresses of pastebin and quickly came across the next limit on the speed at which I was downloading.

Pastebin complained stating that I was hammering them pretty hard and then I had to introduce a time lag with induced sleep of 1 second for every paste download to comply with 10k  requests every 10 mins. It took me around 7 days, 7 whole days to download the entire set of addresses. Quick glance at the download shows there were some trends in the sizes of the files downloaded and most frequent one was only 33 bytes.

Opened up a 33 byte file and it states "Error, we cannot find this paste.". Aha, then it struck me that pastes are not going to be living for ever and it is up to pastebin or creators of the pastes to decide the life and hence the 33 byte responses. Out of the 500K, 95,311 were not downloaded because either they were deleted for violating the policies of pastebin or deleted by original paste creators.

Anyways, with this entire one week of scouting/scraping activity led me to another couple of millions plain text  and unique passwords. Well, I got to state that the various forms and styles in which passwords are exposed in pastebin is one hell of a task to handle. Luckily Python saved me again and with the help of good regex and parsers, I was able to overcome them within a day. Point to note here is that, it is against the policies of pastebin to share credentials however there are still enough pastes which still contain plain text passwords available for all to view/search use or misuse.

If pastebin is interested or anyone from pastebin is reading this or can provide me with a contact, I am more than happy to share those addresses for remedying this open exposure.

Now that we have a good collection of  ~850 million real time passwords, next logical step is to create something useful for all rather than sit in one of my private folders serving none. Created a simple one page search and populated this using response from API for quick connectivity.

 With the help of zxcvbn (Password Strength Estimation tool), we can further strengthen the password field strength for the benefit of all. For added benefit I have also thrown in an additional 1 billion plus words from a famous wordlist for adding further strength to the process. Roughly both put together we get 1.85 billion words to be checked against and in my humble opinion should be sufficient for us now and in the near future.

Alright enough said, let us check few of the screens captured for the benefit of all.

Technology & Architecture Used : 

I want to reserve the technology and architecture for a detailed blog post. However want to provide you with the basic details of technologies used and architecture deployed for hosting XoN.

My entire infrastructure requirement for this activity is only a single instance of Linux running Ubuntu with 1  vCPU and 4 Gb RAM in Google Cloud. From a cost benefit stand point and also depending on the infrastructure provided running the most used website Google, I think I can depend on them for long. Since it runs on GCP, I have also enabled auto-scaling to meet demands during peak usage or heavy use.

From a technology stand point, I have only used custom Python scripts for  downloading, sorting, inserting  and parsing all the data downloaded to extract the passwords. Python has given me the flexibility and speed to code these functionalities as well as excellent supporting libraries for each and every requirement needed as part of this exercise.

Same goes for API as Python is the base on which I have build the API which is the main engine powering the search function and returning results in microseconds.

Entire data is stored in Google Datastore for lightning fast search results needed in our requirement. Cloud Datastore is a highly-scalable NoSQL database which automatically handles sharding and replication, providing us with a highly available and durable database that scales automatically to handle our applications load thereby giving us infinite scope for scaling.

Privacy Requirements :

Well, well with all these exposed passwords and using this as a source for XoN services raises the next big question on how do I handle privacy.

First point to mention is that none of the passwords are stored as plain text instead they are stored in SHA3-Keccak 512  which is a 128 byte one way hash. Currently there is only one tool which has even implemented cracking keccak hashes.

The idea is to avoid easily available MD5 / SHA1 dictionaries for easy brute forcing. I am at peace that I have raised the bar sufficiently.

Second point is that none of the passwords need to be sent directly as plain text to XoN irrespective of their usage through website or API. Only SHA3-Keccak 512 are accepted as input.

Third and last point to note is that I have also inserted anonymous searches as part of API for everyone interested to make use of this functionality. The idea is to send only the first ten characters of the 128 byte one way hash to be searched and the API will search the database only with the shared characters.  If you look at the the set available with 10 characters we are having a huge value to the tune of 16 ^ 10 . This huge space also minimizes the clash of hashes beginning with the same 10 characters thereby giving close to accurate results even when using anonymized searches.

This also avoids majority of the concerns from folks who might be worried even about sharing a one way hash to the API for search.

Tor for XoN !

For privacy conscious user, I have even deployed the same as a Tor-onion site and it is reachable at xposedwxxe7bbqlz.onion

Hashes for Download !

Of course, the entire bunch of hashes will be available as torrent shortly. Based on the feedback and interest, will decide on the timeline for the torrent launch.

Last, but not the least please visit https://XposedOrNot.com to find few more interesting pages to feed your appetite on related breaches.

Feel free to share in your feedback and comments down below for the benefit of all .

Update 05-Jul-2018: Cracking tool for Keccak hashes updated based on input from @s3inlc

21 May 2018

Basic Hygiene & Precautions for Better Password Security

What should I do when my email/passwords are breached and xposed ?

  1. Change your passwords immediately to a unique one
  2. Also change passwords in places where you have used this password ( the xposed one )
  3. Enable and use 2FA (Two Factor Authentication) or MFA (Multi Factor Authentication). In simple terms, it is the use of enforcing an additional layer like an SMS or application based OTP (One Time Password) to be generated uniquely for that login. Most of the commercial websites have enforced 2FA/MFA for their customer logins.
  4. Check with your immediate circle of friends & family and help them change the credentials. Every act of caring and supporting each other is always good for relationships and business.
  5. Please make use of a password manager  and enforce strong password for managing the same .

How to choose good passwords

Try these for your passwords :

  • It is recommended to have any password nothing less than 12 characters in length ( alphabets, numbers & special characters )
  • In one of the password related blogs I came across recently, it was even recommended to keep it much longer than 12.
  • Most of the current password rules currently do not have any major restrictions on numbers. Hence make good use of that opportunity and fully utilize for safeguarding yourself.
  • Password managers can also be a helpful tool in saving all the passwords in a controlled fashion.

Avoid using the below while creating passwords :

  • Strongly avoid using the same password in any of the websites or services you use. This helps minimize the impact, even if one website is breached or impacted.
  • Avoid using words commonly found in any dictionary.
  • Avoid using personal information like kids name, date of birth, city of living as your password
  • Avoid easily guessable words and sequences. For example 123456, love,abcde & password are already in the top 10 most frequently used and abused passwords. Hence it is prudent to avoid them at all costs.
  • Avoid using your login or user name in other forms like user name,capitalised, reversed etc as your password. These are generally the first steps tried by attackers targeting specific users.

Suggestions to be used while selecting your passwords :

Use Common Phrases
-- "Never ever use a single password in multiple sites !"
-- "I love my puppy Milky for he is my life :D"
-- "Do you think you can read my password so easily ???"

Feel free to make your own statements and use them as your passwords. This is much safe and easy to remember as they will be unique to you.

Well, why not add some numbers and make it even more difficult .

-- "Never ever use a 1 p4ssword in multiple sit3s !"
-- "I l0ve my puppy Milky for he is my lif3 :D"
-- "Do you think you can read my p4ssword so 3asily ???"

Oh well , alphabets and numbers are used , what about the special characters .....

-- "Never ever use a ! pa$$word in multiple s!tes !"
-- "I love my puppy M!lky for he !s my life :D"
-- "Do you think you [email protected] read my [email protected]$$word so [email protected] ^-^"

This looks so simple and easy, lets further spice it up

-- "Nevah evah use a 1 p4ssword in mltple sit3s !"
-- "I luv mie ppy Milky for hes mieeeh life :D"
-- "Do u thnk u cn rd mi paswd so 3sly ???"

Why not mix them all and see what happens ?

-- "[email protected] [email protected] use @ ! [email protected]$$word !n mltple s!te$ !"
-- "I luv m!e ppy Milky f0r h3s m!eeeh l#f3 :D"
-- "D0 u thnk u cn rd m! [email protected]$wd s0 esly ^-^"

Let us also learn from the wisdom of all compiled here. Here are the most used passwords and better to avoid these for any of your accounts/transactions. 


123456 123456789 admin starwars hello
passwordletmein welcome 123123 freedom
123456781234567 monkey dragon whatever
qwertyfootball login passw0rd qazwsx
12345iloveyou abc123 master trustno1

It is up  to your imagination to create safe and simple passwords for your use while making it extra difficult for others to guess it.


  Famous words of an infamous person :

Passwords are like underwear, always keep it hidden and safe.
Passwords are like underwear, better to change it often to be clean.
Passwords are like underwear, not to be shared with friends unless you want to get their infections too.

Jokes apart, safeguarding and maintaining password hygiene will help us a lot in avoiding unwanted issues. 


 Be Safe and Stay Happy 😊 !