I found an old Slashdot bookmark of mine, from mid-2010.
Conception of the 1950's of the future, from a 1925 point-of-view. May be old news to Slashdot, as it was already Dugg 3800 times. It is new to me though, and made me feel a little sad. It was almost utopia, viewed through a glass darkly. Looking backward, Bellamy?
Popular Science MonthlyI think that "Dugg" referred to the old Digg versions 3.0 and 4.0, prior to being ruined by then-owner Kevin Rose. I think that is name. It is still a cool image, and not dark at all. Click through to see full-sized, at the source.
We apply the FLUSH+RELOAD side channel attack based on cache hits/misses to extract a small
amount of data from OpenSSL ECDSA signature requests. We then apply a “standard” lattice technique to extract the private key, but unlike previous attacks we are able to make use of the side-channel information from almost all of the observed executions. This means we obtain private key recovery by observing a relatively small number of executions, and by expending a relatively small amount of post-processing via lattice reduction. We demonstrate our analysis via experiments using the curve secp256k1 used in the Bitcoin protocol. In particular we show that with as little as 200 signatures we are able to achieve a reasonable level of success in recovering the secret key for a 256-bit curve...
Using the side-channel attack described, above, 200 signatures were sufficient to recover the secret key, as demonstrated using the encryption protocol (secp256k1) for bitcoin. Now, the authors claim to have refined their approach, thus reducing the number of signatures to 25.
Originally posted by iacr_eprint at Just a Little Bit More, by Joop van de Pol and Nigel P. Smart and Yuval Yarom IACR eprint 2014/434 [PDF]
We extend the FLUSH+RELOAD side-channel attack of Benger et al. to extract a significantly larger number of bits of information per observed signature when using OpenSSL. This means that by observing only 25 signatures, we can recover secret keys of the secp256k1 curve, used in the Bitcoin protocol, with a probability greater than 50 percent. This is an order of magnitude improvement over the previously best known result. The new method of attack exploits two points: Unlike previous partial disclosure attacks we utilize all information obtained and not just that in the least significant or most significant bits... Furthermore, whereas previous works require direct information on ephemeral key bits, our attack utilizes the indirect information from the wNAF double and add chain.
Emphasis mine. Candid disclaimer: I have not read either paper.
For a more detailed understanding of bitcoin vulnerability, see Dispelling some myths about Bitcoin (serious). For an intuitive understanding of bitcoin cultural approaches to attack vulnerability remediation, see Novel method for backup of wallet seeds or private keys (not-so-serious).
There has been so much tumult in bitcoin and crypto currencies over the past few days! Interest and concern extends far beyond arcane online communities. Motives vary.
Decentralized and anonymous
There are two conceptual pillars of trust that uphold bitcoin as being superior to fiat currency. The first is decentralization. The fiat currency of reference is primarily the US dollar, for the time being. Why? Because the $US is the world's reserve currency, for now. If Germany weren't part of the EU, if Japan weren't still in its lost decade and England weren't so afflicted with problems, the Deutsche mark, Yen or British Pound Sterling would be attractive alternatives to the $US as a fungible, stable store of value. The Yen is still a good currency. As ideological (not market) confidence in the $US has diminished, the appeal of an apolitical, modern alternative increases. I won't go off on a tangent as to why a currency printed by the U.S. Treasury in support of the Federal Reserve's monetary policy isn't as highly esteemed as it was in the past. Obviously the $US dollar is a highly centralized currency.
The second conceptual pillar of bitcoin is anonymity. US dollars held as cash will be anonymous until one wants to use them for exchange for commercial transactions of size. Bitcoin has some anonymity short comings, too, but they may be tractable remedies.
( Read more...Collapse )
Centralization of bitcoin
All markets are game theoretic. Bitcoin is more transparently so. (I really wish we could ask Professor John Nash what he thinks of bitcoin! Nash actually wrote a pleasant, accessible article about bitcoin-like currency a few years ago.)
I mention game theory because bitcoin's most acute concern now is loss of decentralization. It is due to the documented, persistent existence of a 51% majority mining pool controlled by gHash.io. gHash is owned and operated by a private entity, cex.io. gHash's market dominant behavior was noted in March 2014, but the situation was transient, unlike now. Monopolists (and cartels!) can assert control as a function of mining power. See How a mining monopoly can attack bitcoin for a chart of strategies that mining pools can pursue as a function of hash power. it was recommended by Ed Felten in his post, Bitcoin mining now dominated by one pool.
Production and transaction costs
In theory, bitcoin is a perfectly smooth, zero transaction cost medium of exchange. In reality. this is possible but involves a modicum of effort. Some bitcoin miners and many many non-mining users keep their holdings in custody of a clearinghouse such as Mt. Gox or Silk Road. This does generate small transaction costs. In return, one benefits from the greater convenience in making purchases and sales. The clearinghouse may have an additional appeal to miners, as it offers the option of participating in a shared mining pool. Bitcoin was designed to reward early adopters; as more bitcoins are mined, more computational effort is required.
Mining bitcoin requires processing power and electricity. In 2010, a PC with an NVIDIA or ATI GPU would have been adequate, but no longer. A new crypto currency-specific manufacturing industry has evolved for bitcoin mining equipment, using FPGAs which are more energy efficient than graphics processing units. This was further improved by an application-specific integrated circuit (ASIC),
- in other words, a chip designed from the ground up for the specific purpose of mining bitcoins. ASIC also represents the theoretical limit on the hardware capabilities of mining equipment.
Electricity and and misbehavior costs
Mining bitcoin is costly due to mining equipment prices and the high power requirements, which result in large amounts spent on electicity.
Perverse incentives motivate uneconomic choices. The most egregious and harmful behavior (directly associated with bitcoin mining) that I've seen to-date was unauthorized use of a National Science Foundation supported supercomputer to mine bitcoin. $150,000 in computing resources, e.g. electricity, were spent in order to mine the equivalent of $8000 in bitcoin. Another incident occurred at Harvard University in March 2014. The researcher used Harvard's high-powered network of thousands of CPU cores to mine an unspecified number of dogecoins.
Walgreen's is already under pressure from hedge fund investors that want it to move across the Atlantic. Walgreen's management doesn't want to do so, but they may not have any choice.
Senator Carl Levin said companies that moved overseas were unfairly taking advantage of many of the benefits of doing business in the United States but not paying their fair share of taxes. Those benefits, he said, include “patent protection, research and development tax credits, national security and more.” He added, “They shouldn’t be allowed to shift their tax burden onto others.
via The New York Times, Pfizer's Plan to Go to London Spurs Other Companies to Consider Moving Abroad
My favorite reactions
Hint: At least one is mine.Will this relocation make all Pfizer and AstaZeneca drugs ineligible for purchase by the US government or programs like Medicare and Medicaid? And make these firm ineligible for R&D funds from NIH, NSF and other agencies - either directly or as "industry partners" on academic grants? I hope so. Lose US taxes - lose access to US market. It's only fair.
- Perhaps the FDA should get on the ball and tell Pfizer that if it becomes a British Company it will have to spend tens of billions to have all its drugs reconsidered for being sold in the USA, Are previous licenses would be null and void.
We should impose tariffs on any company that does this. If it doesn't cost them to do it they will.
- Tariffs sound like a great idea to me! I have had enough of globalism. Thanks to free trade, we've outsourced our R&D, defense and manufacturing overseas. China and India don't want AWS nor Amazon retail in their domestic markets. Good for them! Walgreen's wanted to stay here in the USA, but activist shareholders are soooo persuasive. Do any other activist investors even exist, besides Messrs. Icahn, Loeb, Einhorn, Ackman and T. Boone Pickens?
The most depressing thread of allThey should have to go through much different FDA approval processes to be able to get approval to market their foreign made drugs in this country.
- These are multinational corporations. They now have facilities all over the world, and research done in the U.S. will continue to be supported not only by the U.S. government but by the U.S. our educational and research infrastructure. Pfizer has only to move some of its bureaucrats to the UK. All information, research and business can be in the cloud.
- They now only need to secretly donate to 501c groups. And now-- amazingly!-- Bitcoin! Election commission votes to allow bitcoin donations. Both completely anonymous. Makes you wonder who is really funding our campaigns now.
It wasn’t supposed to be this wayGlobalization's New Underclass
Globalization has long been portrayed as the rising tide that lifts all boats. The surprise is in the tide -- a rapid surge of IT-enabled connectivity that has pushed the global labor arbitrage quickly up the value chain. The rich are, indeed, getting richer but the rest of the workforce is not. This spells mounting disparities in the income distribution...
I wondered, "Who wrote this?" Surprise, again! It didn't run in The Nation, nor The World Socialist Worker website. Instead, the author is Stephen S. Roach, Managing Director and Chief Economist of Morgan Stanley Dean Witter. He wrote it in March 2006.
None are sparedThe malaise is not limited to unskilled labor and the manufacturing workforce.
The Internet has forever changed the competitive climate for most white-collar knowledge workers: software programmers, engineers, designers, professionals toiling in legal, accounting, medical, actuarial, consulting, and financial-analyst positions -- sparing an increasingly small portion of those at the very top of the occupational hierarchy. Washington’s penchant for cutting taxes of the wealthy probably hasn’t helped matters either.
These observations about China extend to the United States.
With per capita income of $38,000 and $1,700, respectively, the United States and China are at opposite ends of the spectrum. Yet both countries have extreme disparities...the poor [in China] now have real-time access to the “outside world”... the Internet has the potential to spark resentment and social instability...
At the time, the author's greatest concern was increased protectionism. Ultimately, he may be correct, I don't know. It is now November 2013, eight years later. What will resolve this incipient instability?
Inequalities of the income distribution have long been the Achilles’ heel of economic growth and development...that seems more the case than ever. History tells us that the pressures of widening income disparities are often vented in the political arena.
This is Lady Pine's follow-up to her post about peculiar AWS pricing anomalies that I was so excited about at the time.
It seems like Amazon changed their reserve price algorithm last July, using a non-symmetric algorithm instead of the AR(1) up-down symmetric algorithm, and changing floor and ceiling values. Then in October it seems like they returned to using a simple constant minimal price algorithm, with yet a new level of a minimal price!
Was our paper a reason for the abandoning of the (no longer) hidden dynamic reserve price?
I found a video on YouTube while browsing, randomly. Maybe. YouTube is a Google digital property. I don't try to hide from Google. Their personalized advertising knows me too well. The video is near-future fiction, about the destiny of newspapers.
Newspapers have not done well during the past 12 years. This chart indicates that recent advertising revenue has declined to 1950's levels, although I do not know if there is any adjustment for inflation.
2015 GoogleZonThe title is Zeitgeist-and-angsty. Try not to be put off by it. 2015 GoogleZon begins with the premise that by 2010, Google is the Internet's dominant entity. It influences all information flow, though it does not actually create content, as algorithmic journalism applications claim to do.
Time passes. The world changes. The New York Times goes off-line in 2014, publishing a weekly print edition. The only customers are wealthy, Upper West side residents, and the elderly. A former New York Times journalist, Pinky (referring to feminine identity, or Communism?!) initiates a crowd-sourced relay network of GPS-tagged audio updates. Her communications network is primitive, but well received. The video ends with Pinky and her emergent phenomena that might be hinting at citizen journalism.
Digression: New media
Citizen journalism has generated excitement as an alternative to traditional news media. I first came across the idea in 2009, after reading about SXSW in Austin, Texas. My online acquaintance, @znmeb, develops open-source computational journalism software. It is designed to facilitate credible reporting, with robust data analysis methods. Intended users are professional and citizen journalists.
Blogs are another relatively recent form of non-traditional media, offering current events analysis, movie reviews, historical retrospectives, educational materials and photographs. Forbes Online and The Huffington Post are non-traditional media publishers. I prefer Forbes, as their contributors are better. Arianna Huffington has a reputation for paying writers poorly, or nothing at all. That probably accounts for Huffington Post quality issues! The decent reporters leave, in order to earn a living wage. There are smaller, very active specialty websites, such as StockTwits, reddit, Seeking Alpha and Slash Dot. Other platforms, e.g. Twitter and identi.ca, enable terse real-time reporting by the public.
Google zone reduxCitizen content creators and traditional news media have an uneasy relationship. The first comment on the GoogleZon video captures that conflict:
Nothing but a collection of trivia, much of it untrue, all of it narrow, shallow and sensational - That's what The New York Times thinks of blogger journalists, according to this!
This reminded me of Eli Pariser's "filter bubble", about confirmation bias and its consequences:
The whole issue of providers feeding you things "you like" is sort of like inbreeding. We only see stuff we've already seen before, and soon everyone is pigeon-holed into a deficient world view of their own making. There is no new DNA in your thought patterns and soon the genetic mutations rise to the top and we're left with a world full of retarded ideas.
And in this new world, who will dig out the facts on which stories might be based? Or will we all be relying on machine generated gossip? Will Pinky even be able to pay her rent?
NowConcern about Google was valid, but not directly relevant. Instead, Jeff Bezos, majority shareholder and CEO of Amazon, bought The Washington Post for $250,000,000, as an all-cash transaction. His purchase is a surprise, particularly given his vision for newspapers:
About one thing I am certain: In 20 years there will be no more printed newspapers. If they do exist, they will be considered a luxury item for certain hotels to offer as an extravagant service for their guests. Printed newspapers will no longer be normal in 20 years.
Emphasis mine. Recall that dystopian future envisioned for newspapers, in 2006: the only customers of the New York Times printed edition would be the wealthy and the elderly. Mr. Bezos's stated intent for The Washington Post is to use it as an experiment.
News of the WorldRupert Murdoch comes to mind. Murdoch is 81 years old. He continues to have a strong interest in publishing. The newspaper business was possibly his first, maybe only love. In contrast, Jeff Bezos is 50 years old. According to Mr. Bezos's spokesman, his only prior interest in publishing or news media is his fondness for reading books. Ownership of a major newspaper offers comprehensive access to the public, for purposes of commerce and political influence. Rupert Murdoch was limited by 20th century technology. Now, with the Internet, satellite communications and ubiquitous GPS, Jeff Bezos will not be as constrained:
During Murdoch’s heyday, it was impossible for one company to consolidate production, distribution, editorial and advertising functions across all forms of media and across national borders. The internet makes that, if not easy, for the first time, possible.
- We all know what kind of newspaper owner Jeff Bezos will be, Corey Pein, 6 August 2013
Amazon.comThe following anecdote about Amazon cost control methods at its Allentown facility in 2009, verified by Morning Call of Pittsburgh and The New York Times, is disturbing:
Instead of paying for air-conditioning at some Pennsylvania warehouses, Amazon had just stationed paramedics outside to take the inevitably heat-stressed workers to the hospital.
chimera of profitless growth. After 16 years, Amazon has never paid any dividend (of consequence) to investors; however,
- Amazon and Walmart have exacerbated our new, low-wage economy;
- Jeff Bezos used Amazon's market dominance to devastate the publishing industry and
- more successfully than any other e-commerce entity, Amazon cultivated a "faster-cheaper-further" mindset that is eroding communities.
Further developmentsI overlooked this week's other major newspaper-related event. Jeff Bezos, formerly of hedge fund D.E. Shaw, was not the only billionaire to purchase a prominent media company. On 5 August 2013, John Henry purchased the Boston Globe for $70 million. John Henry is also a prominent financier, especially well-known for his pioneering work in high frequency trading.
On 7 Aug 2013, New York Times "not for sale" Say Owners. The Koch Brothers decided not to purchase The Tribune newspapers a few weeks later.
* Photograph via Seeker 401
The Enlightenment is also known as "The Age of Reason" because of the remarkable advances in science and rational thought that are associated with the 200 years between 1650 and 1850. I always believed that we unequivocally benefited from the Enlightenment, but I just learned of an alternative point of view.
There are some portions of this post about the Enlightenment with which I am uncomfortable. Be that as it may, it is worthwhile to learn from the history cited.
N.B. In my post, every indented and block quoted item that is excerpted from the Simple To Remember website written by Aish-affiliated Rabbi Ken Spiro unless indicated otherwise. Emphasis mine, throughout.
One of the casualties of The Enlightenment was organized religion. Religion was perceived as an intellectual shortcoming, like superstition, rather than science. Secular culture is the Rationalist alternative to religion. Given that, I thought this conclusion was peculiar:
As odd as it may sound, the less religious the Western world became, the better it treated the Jews.
Shouldn't they treat us better, not worse? Why is it odd?
Civil rights for Jews
Jews were specifically granted some basic, though still not equal rights in 1791. Per the French National Assembly:
The conditions requisite to be a French Citizen are fixed by the constitution...every man who takes the civic oath...has a right to all the advantages it insures; all adjournments, restrictions, and exceptions affecting individuals of the Jewish persuasion are annulled for those Jews who shall take the civic oath.
It sounds very fine! I could not conceive how it could be objectionable, yet this was Aish's view:
Judaism believes that for an ideal world there must be a focus on both God and man. Because without a focus on God, all moral values become relative.
I don't know what causes moral values to become relative. I do know, with certainty, that moral relativism is bad. I have seen evidence of this:
For a while, it might be nice to have respect for civil rights, but when it becomes convenient or necessary (for various social or political reasons) to change that focus, then respect for human life becomes just another idea that goes out of style.
Napoleon and the Jews
I did not realize that Napoleon Bonaparte marched through Europe, liberating Jews from their ghettos, wherever he could. Napoleon said,
I will never accept any proposals that will obligate the Jewish people to leave France, because to me the Jews are the same as any other citizen in our country. It takes weakness to chase them out of the country, but it takes strength to assimilate them.
Napoleon's edicts, initiated during the late 18th century, were successful, wonderfully so. By the late 19th century, Jews - who had been economically and physically marginalized, who had been locked out of the trades and professions - now were allowed (if not exactly welcomed) into all phases of European society.
I think Napoleon meant well. He was suffused with the good spirit of the French Revolution, of liberte rather than with the awful part, The Terror. I also realize that Napoleon Bonaparte was on the very endpoint of the interval described by mainstream sources as The Enlightenment. Aish was correct in noting this detail:
He [Napoleon] wanted the Jews to be accepted by the rest of European society...So, he set about to help the Jews rid themselves of the things that set them apart. Napoleon was asking the Jews to answer the great question that came out of emancipation: What is your primary identity? Are you first and foremost Jews or Frenchmen?
The distinction was not resolved. It eventually became critical, though not in France.
Jews and the Industrial Revolution
This rings true:
Once the gates of the ghettos were thrown open, the Jews rose to the top quickly, gaining prominence and wealth.
It is consistent with what my grandparents told me. This too:
In theory Jews had equal rights but in practice, the story was very different. Many Jews saw conversion as the best way to advancement in enlightened Europe.
Jews were accepted into society, just as long as they were not too Jewish.
The term anti-Semitism came to prominence during the late 19th century. It is attributed to Wilhelm Marr, a German writer, from a pamphlet he published in 1879. Anti-Semitism was intended to differentiate between antipathy for Judaism as a religion versus antipathy for Jews as a so-called race or nation.
Jews in fin de siècle Germany
In Germany, Jews prospered visibly. They contributed to German industry, commerce, the arts and scientific discovery. Be that as it may, the "writing was on the wall" (or should have been, that all was not well) when The Question of the Jew is a Question of Race, by Karl Eugen Duehring (1881), was well-received throughout Germany. Many copies were sold, over the next two decades, equivalent to what we consider to be a best seller today. The book's conclusion was that the distinction between Jewish religion and Jewish race is false. Instead, they are one and the same, inextricably linked. The logical consequence is that it is irrelevant whether Jews consider themselves to be Jewish. That viewpoint remains relevant in the 21st century, regardless of whether it is accurate, preferable or otherwise.According to Karl Duehring, and the more formalized anti-Semitism that followed, it doesn't matter if Jews convert to Christianity. Being non-secular or striving to contribute to the community, culture, including military service, is of no significance. Anti-Semitism as a social and cultural ideology did not care what the Jews of Germany believed about their status as former Jews, nor how non-Jewish they behaved. All that mattered was that they were identifiable as Jews.
Jewishness was determined by anti-Semites in Nazi Germany, not by Jews.
Rousseau versus LockeThe central premise of the Aish essay was that it would have been better for the Jews if Napoleon and France had not been a dominant influence in western Europe and later, in the United States, because it reinforced the belief that non-secular life could be a legitimate and fulfilling choice for Jews. Aish's view is that a life without a place for God would lead to moral relativism. Instead, Aish thought that John Locke's belief in the need for God in public and private life was superior.
Maybe the Jews are no different than the Puritans of Max Weber's The Protestant Work Ethic, i.e. it is more difficult to endure in times of freedom and wealth than persecution and poverty.
What is my point?So, why bring this up now? No, I do not believe that Germany will do it again! I find Germans to be more respectful, receptive and friendly, in their formal way. They will praise me for logical thinking, and in economic or financial discussions, unlike the American, UK and most Israeli Jews with whom I interact.
Even though some of the Aish post is not consistent with my view, it described the underlying sensibilities of non-Jews and Jews in nearly half of the 20th century, in Europe. I have been troubled by the prevalence of anger and hostility toward Jews recently. I am saddened by anything that is against Israel, but I fully understand the rationale. It is certainly based on actual events and conflicts of the recent past. Resurgence of interest in the Elders of Zion, or revisionist history espoused by well-educated citizens of developing nations is especially troubling.
Distrust of financiers, corporate executives and investment bankers is everywhere. That is a logical reaction. It is unlikely to dissipate until the underlying cause, namely, vast concentration of power and wealth with a tiny group of aloof elites, lessens. Many of these individuals with the most visible public profiles are Jewish, or formerly Jewish. THAT is also deeply troubling. I am acquainted with one of these individuals. I tried to mention my concerns to him. He was dismissive of me, and my supposedly simple-minded understanding of the world.
IsraelJews worry about Iran, Syria and Palestinians. Unless there are significant changes, Israel's enemies won't need to act, merely wait for civil war in Israel. That is another of my concerns.
Ultra-Orthodox Jews in Israel are sheltered and indulged, e.g. exempt from military service, modern work responsibilities. I am reminded of my family. My great-grandparents had seven children, but my great-grandfather never worked. He spent all day in the synagogue praying, while my great-grandmother used her dowry money to start a dry-goods store. She did the cooking, cleaning and much of the child-rearing, while supporting the family financially. When the children were old enough, ten or twelve years of age, they worked in the store after school. My great-grandfather continued to pray all day, yet he was considered a good husband. I never heard a word of complaint about him from any member of my family. Will non-Orthodox Jews want to fight for and sacrifice for the ultra-Orthodox, e.g. their large families, the settlements, forever?
The schism between the women who want to pray every day at the Western Wall, versus those who don't want them to do so, is terribly polarizing. The devoutness of the women who insist on prayer at the Wall, in public, is new. My perception is that Jewish women are NOT a repressed underclass, anywhere. (Note, though, that many cultures have a superficial appearance of meek or submissive women. However, in the household, and other situations that are not part of public life, the reality is very different!) I worry that the controversy over women praying at the Wall, which is extreme on both sides, including Haredi Jews hurling epithets and eggs at the women, might escalate such that it could endanger Israel and Judaism. This becomes a more plausible outcome when combined with the long simmering irritation felt by working, non-ultra-Orthodox Jews about the non-working, ultra-Orthodox Jewish settlers.
In the Land of the FreeI cringe every time I read about some new example of corruption, vice or financial excess, as there are so many Jewish names associated. I would prefer that NONE be Jewish. A tiny sample:
- Robert Filner, who served 10 terms in Congress. He is the first Democratic mayor of San Diego in 20 years. Rather, he was the mayor, "San Diego Mayor Bob Filner stepped down... after the City Council voted to accept his resignation as part of a settlement deal. Eighteen women have so far accused the Democratic mayor of sexual harassment." - via Politico, San Diego Mayor Bob Filner Resigns
- Anthony Weiner, former mayoral candidate for New York City
- the hedge fund industry, most egregiously Stephen A. Cohen of SAC, although nearly all of the most ostentatious nouveaux riche hedge fund managers are "of the Jewish persuasion", as the French General Assembly of 1791 would have said;
- the Wikipedia category, American Jews. There is an Atheist Jews category too. Both read like a rogue's gallery, which breaks my heart.
Regarding unfamiliar Jewish-themed websites and doubts about AishOften when I am online, I run across "Messianach Jews", sometimes known as "Jews for Jesus". Some do not state that, which confuses and irritates me, once I figure it out. If I am reading a religion-oriented website post, I want to know the affiliation! I don't care if the objective is proselytizing, or if the site (and religion) is hostile to non-believers. I don't like behavioral manipulation and deceit.
The New York State Comptroller recently announced that 2012 aggregate Wall Street bonuses are $20 billion, an 8% increase over last year. Oddly, the average salary for a Wall Street worker is more than it was prior to the 2008 financial collapse. It is also five times greater* than what most workers in New York City earn.
Consolidation of wealth and income doesn't spare the financial services sector. I've said that for awhile. Finally, here's a shred of proof. Employment has dropped 10%, a loss of approximately 20,000 jobs, in the Wall Street job market due to company downsizing. Yes, there was more money made last year, but for fewer people. For further depressing details, see Wall Street Bonuses Are Up 8%! Happy Hump Day! via Village Voice (27 Feb 2013) and ABC News.
* The comparative earnings data is from 2011, so I'd expect 2012 to show even greater disparities. After all, we are in the midst of an economic recovery. Correct?
It's almost impossible to keep a web server secret by not publishing links to it. As soon as someone follows a link from your "secret" server to another web server, your "secret" URL may appear in the referrer tag and can be stored and published by the other web server in its referrer log. Similarly, the web has many outdated and broken links. Whenever someone publishes an incorrect link to your site or fails to update links to reflect changes in your server, Googlebot will try to download an incorrect link from your site.
How to block search spiders from crawling your websiteIf you have pages or other content that you don't want to appear in Google's search results, consider these two options.
- Most straightforward: If you need to keep confidential content on your server, save it in a password-protected directory. Googlebot and other spiders won't be able to access the content.
- Use robots.txt to control access. Fine tune by file type and directories if you want. The robots.txt file tells Googlebot and other crawlers what parts of your website not to crawl for indexing.
Most morally upstanding robots will respect the directives in a robots.txt file. Others may not. The directives of a robots.txt file are NOT legally enforceable! Don't forget spammers either! They won't be stopped by a robots.txt file.
Robots meta tagsThe way to prevent Google from including a webpage anywhere in search results is to use a noindex meta tag. Once Google crawls the page, it will completely drop it from search results. It will not be included, even if other pages link to it. Other search engines might behave the same, or differently than Googlebot.
According to Webmasters — Google Developers, via archive.is: Robots meta tag and X-Robots-Tag HTTP header, specifications:
When robots.txt is unnecessaryYou need a robots.txt file only if you do not want search engines to index part of the content of your website.. If you do want search engines to index everything in your site, you don't need a robots.txt file, not even an empty one.
The Google /robots.txt checker and analysis tool (accessed via Google Analytics) will be retired soon. There are alternatives for creating a robots.txt file though I don't understand why Google is discontinuing theirs, along with so many other services, recently.
For reference, and maybe mere curiosity about the range and variety of major but mostly minor crawlers, the Web Robots site maintains a user and/ or owner submitted Robots Database of robot software implementations and operators.