Get the last effective URL from a series of redirects for the given URL

As some of you may know websites like Facebook or Twitter allow you to fetch user profile images if you know id of the given users. For example, you can fetch the Facebook image for Mark Zuckerberg by going to http://graph.facebook.com/4/picture (4 is Mark's user id), however if you pay attention to the address bar you will notice that the the aforementioned URL actually redirects to http://profile.ak.fbcdn.net/hprofile-ak-snc4/157340_4_3955636_q.jpg.

Same goes for Twitter - I could fetch my user picture by going to http://api.twitter.com/1/users/profile_image/codeaid but it actually redirects to https://si0.twimg.com/profile_images/321201130/twitter_bigger_normal.png.

In one of the projects I've been working on lately, I added a functionality for people to log in using their Facebook, Twitter and other service credentials. I am also storing URLs of their user images in the database for displaying on the comment pages. As you saw earlier it is easy to generate URLs redirecting to the correct destination, however as you can imagine it doesn't make sense to store them as they are as you would have to execute those redirects every time you access the image. Imagine if you have a page with comments and every comment has a picture - it could and would take a long time to resolve and load them all.

That lead me to writing a simple function, which takes the starting URL, follows all the redirects and returns the last effective URL for it. From examples above the last effective URL for http://graph.facebook.com/4/picture would be http://profile.ak.fbcdn.net/hprofile-ak-snc4/157340_4_3955636_q.jpg.

The function looks like this:

  1. <?php
  2.  
  3. /**
  4.  * Get target url from a redirect
  5.  *
  6.  * @param string $url Source url
  7.  * @return string
  8.  */
  9. function getLastEffectiveUrl($url)
  10. {
  11.     // initialize cURL
  12.     $curl = curl_init($url);
  13.     curl_setopt_array($curl, array(
  14.         CURLOPT_RETURNTRANSFER  => true,
  15.         CURLOPT_FOLLOWLOCATION  => true,
  16.     ));
  17.  
  18.     // execute the request
  19.     $result = curl_exec($curl);
  20.  
  21.     // fail if the request was not successful
  22.     if ($result === false) {
  23.         curl_close($curl);
  24.         return null;
  25.     }
  26.  
  27.     // extract the target url
  28.     $redirectUrl = curl_getinfo($curl, CURLINFO_EFFECTIVE_URL);
  29.     curl_close($curl);
  30.  
  31.     return $redirectUrl;
  32. }

As you can see I am utilizing cURL library for this purpose. First, I am initializing it with the specified starting URL and setting the following two options:

  • CURLOPT_RETURNTRANSFER meaning that the output from the call will be returned rather than printed on the screen and...
  • CURLOPT_FOLLOWLOCATION, which is a bit more important in this case and as php.net says will "follow any "Location: " header that the server sends as part of the HTTP header" and that is exactly what we need.

You can also set CURLOPT_MAXREDIRS to the number of maximum redirects you want to follow if you wish, however it is usually not needed. You can find more information about the available parameters on the curl_setopt page if you want.

What we do next, is simply fetch the cURL request and return null if it fails. If not, we fetch and return the last effective URL using the curl_getinfo function. Simples :)

The usage is straightforward. If we wanted to fetch the last effective URL for Mark Zuckerberg's profile image we would call the function like this:

  1. $lastEffectiveUrl = getLastEffectiveUrl('http://graph.facebook.com/4/picture');

The value of $lastEffectiveUrl after the call would be the expected

  1. 'http://profile.ak.fbcdn.net/hprofile-ak-snc4/157340_4_3955636_q.jpg';

Hope this helps and, as always, please leave your comments if something is still not quite clear.

Comments
1
You're my hero!

I have seen people ask how to do this dozens of times, and this function is by far the most simple, accurate, straight forward, and effective solution I've seen
Ultravilence, August 28th 2011, 4:05
2
Thanks! Glad to help :)
Andris, August 28th 2011, 18:07
3
On linux - its much easier

curl --head -q http://yourURL.com/webpage.html | grep Location | awk '{ print $2 ;}'
vikash, September 29th 2011, 13:23
4
Hey it could be great if it works, but if you have hosting with safemode On or open_basedir On you won't be able to use it :( I do have safemode Off bud open_basedir (which is absurd in relation to this solution) blocks FOLLOWLOCATION option in CURL :( however there is no other way to do that? But I really need to get it working, do you know how to go around this FOLLOWLOCATION option and do it differently? Thank you so much if you could. Email me if you find any solution. Will be much appreciated! Anyway great tutorial keep going mate.
ArnieX, November 24th 2011, 12:20
5
@ArnieX: I found this comment - PHP: curl_setopt - Manual. Not sure if it helps but it might be something to start with. 

As an alternative, I could suggest Googling this issue. That's how I found the comment - open_basedir AND CURLOPT_FOLLOWLOCATION - Google Search
Andris, December 3rd 2011, 15:11
6
Hey :) thanks for your email, I've found some working solutions even that one from CURL manual but it didn't work well in my case so I did one try with FOPEN and it worked well too it gets whole bunch of redirects and with str_replace it counted how many times http:// was presented and then it took last URL with pregmatch between http:// and empty space. This worked well until few sites has sended ERROR so I've decided to not to go trough this and I've simply implemented API from longURL.org website so this works well for my needs now and I think it is based on CURL too so their webserver is configured differently.

But I use it limitedly for 330+ shortening services not for all URLs what was my goal. But anyway it works well and that is what I need.

I've used it for this Sy.pe - Zkracovač URL / URL Shortener which is URL shortener but I do try to compete with bit.ly and goo.gl so provide as much polished and great working service. I've almost vanished all disadvantages of URL shorteners and some are coming up even these which are still disadvantages for these biggest shorteners.

Before I've blocked attempts to shorten already shortened URLs but now I do unshorten them before shortening or fail as before so that is more user-friendly than just fail without shortening it. In final it saves original URL and that is great advantage of using my shortener than the other :)

Thank you very much for helping me and I hope this would help someone else to solve their problem.

P.S.: If you'd like to see some source codes from me, email me again and I would be happy to provide you some of them. Your website is great and I do appreciate it.

Have a great day.
ArnieX, December 3rd 2011, 15:28
7
Very nice. Exactly what I was looking for. Thank you very much.
Lil, December 26th 2011, 17:25
8
Thanks a lot, but if your server not enable open_basedir and safe_mode, you can use this link to solve

cURL: follow locations with safe_mode enabled or open_basedir set | Slopjong
boypotay, May 2nd 2012, 9:29
9
Thank you very much!
Andrii, July 16th 2012, 15:46
10
Hello, I like this article. Is it possible to  download deleted facebook photos from http://profile.ak.fbcdn.net or their CDN website. I have deleted some pictures but came to know facebook does not delete your picture, it will store it like 'http://profile.ak.fbcdn.net/hprofile-ak-snc4/157340_4_3955636_q.jpg' and remove from your facebook account.

Please let me know if their is any way where we can generat those URL and download our deleted pictures.

Regards,
RaviT
ravi, October 15th 2012, 3:21
11
@ravi - unfortunately that is not possible as the information about the target location of the short graph url is only known to Facebook.

I suppose the issue with them is that if you delete a picture on Facebook, they only delete the reference from their database (e.g. picture_id | user_id | target_url). The image still remains on the CDN. So, once the reference is gone, there is no way of telling what it pointed to. Especially for you as a user.
Andris, October 24th 2012, 10:29
12
Thank u so much :)
Ali, October 9th 2013, 12:50
13
Hello therе! Would you mind if I shaгe your blog with 
my zynga ǥroup? There's a lot of folks that I think would reаlly aрpreciate your content.
Pleasе let me know. Thanks
voyance, March 10th 2014, 21:11
14
Thanks man you saved my day
Raviteja, June 11th 2014, 13:11
Name
Email (required)
will not be published
Website
Recaptcha
you will only be required to fill it in once in this session

You can use [code][/code] tags in your comments