The Internet Archive discovers and captures web pages through many different web crawls. At any given time several distinct crawls are running, some for months, and some every day or longer. View the web archive through the Wayback Machine.
Web wide crawl with initial seedlist and crawler configuration from March 2011. This uses the new HQ software for distributed crawling by Kenji Nagahashi.
What’s in the data set:
Crawl start date: 09 March, 2011
Crawl end date: 23 December, 2011
Number of captures: 2,713,676,341
Number of unique URLs: 2,273,840,159
Number of hosts: 29,032,069
The seed list for this crawl was a list of Alexa’s top 1 million web sites, retrieved close to the crawl start date. We used Heritrix (3.1.1-SNAPSHOT) crawler software and respected robots.txt directives. The scope of the crawl was not limited except for a few manually excluded sites.
However this was a somewhat experimental crawl for us, as we were using newly minted software to feed URLs to the crawlers, and we know there were some operational issues with it. For example, in many cases we may not have crawled all of the embedded and linked objects in a page since the URLs for these resources were added into queues that quickly grew bigger than the intended size of the crawl (and therefore we never got to them). We also included repeated crawls of some Argentinian government sites, so looking at results by country will be somewhat skewed.
We have made many changes to how we do these wide crawls since this particular example, but we wanted to make the data available “warts and all” for people to experiment with. We have also done some further analysis of the content.
If you would like access to this set of crawl data, please contact us at info at archive dot org and let us know who you are and what you’re hoping to do with it. We may not be able to say “yes” to all requests, since we’re just figuring out whether this is a good idea, but everyone will be considered.
TIMESTAMPS
The Wayback Machine - https://web.archive.org/web/20110516011816/http://www.uzdaily.com/articles-id-832.htm
TeliaSonera AB, the leading telecommunications company in the Nordic and Baltic region, has on July 16, 2007, closed the MCT Corp. (MCT) acquisition announced on July 9. TeliaSonera acquired 100 percent of the shares in MCT, a US based company with majority controlling shareholdings in three Eurasian GSM operators in Uzbekistan and Tajikistan and a small minority interest in the leading GSM operator in Afghanistan, for an enterprise value of approximately SEK 2.0 billion (US$300 million).
TeliaSonera AB, the leading telecommunications company in the Nordic and Baltic region, has on July 16, 2007, closed the MCT Corp. (MCT) acquisition announced on July 9. TeliaSonera acquired 100 percent of the shares in MCT, a US based company with majority controlling shareholdings in three Eurasian GSM operators in Uzbekistan and Tajikistan and a small minority interest in the leading GSM operator in Afghanistan, for an enterprise value of approximately SEK 2.0 billion (US$300 million).
"This transaction underlines TeliaSonera’s strategy to be the leading Eurasian mobile operator and strengthens our commitment and presence in the region. Uzbekistan, Tajikistan and Afghanistan, with a combined population of approximately 67 million, growing economies and relatively low mobile penetration levels, provide an excellent growth opportunity for TeliaSonera in Eurasia", says Tero Kivisaari, Head of TeliaSonera’s business area Eurasia.
The four GSM operators are:
Uzbek-American Joint Venture "Coscom" LLC is the third largest mobile operator in Uzbekistan, with 393,000 subscriptions and a market share of approximately 11 percent as of May 2007. Coscom’s net sales in 2006 were USD 26 million and EBITDA US$5.8 million. MCT has a 99.97 percent interest in Coscom. Mobile penetration in Uzbekistan is about 13 percent.
CJSC Indigo-Tajikistan and CJSC Joint Venture Somoncom combined is the second largest mobile operation in Tajikistan, with 377,000 subscriptions and a market share of approximately 27 percent as of May 2007. The combined net sales in 2006 were US$27 million and EBITDA US$13.7 million. MCT has 60.0 percent interest in Indigo-Tajikistan and 59.4 percent interest in Somoncom. Mobile penetration in Tajikistan is about 20 percent.
Telecom Development Company Afghanistan Limited (Roshan) is the largest mobile operator in Afghanistan, with 1,300,000 subscriptions and a market share of approximately 50 percent as of May 2007. Roshan’s net sales in 2006 were US$191 million and EBITDA US$66.5 million. MCT has a 12.25 percent interest in Roshan. Mobile penetration in Afghanistan is about 8 percent.
TeliaSonera AB will consolidate the operations in Uzbekistan and Tajikistan as of the closing. The operation in Afghanistan will be reported as a financial investment.
TeliaSonera is already present in the region through its subsidiary Fintur Holdings, with majority holdings in leading mobile operators in Kazakhstan, Azerbaijan, Georgia and Moldova.
