What you may not know is that the roots for the idea of AWS go back to the 2000 timeframe when Amazon was a far different company than it is today — simply an e-commerce company struggling with scale problems. Those issues forced the company to build some solid internal systems to deal with the hyper growth it was experiencing — and that laid the foundation for what would become AWS.
Speaking recently at an event in Washington, DC, AWS CEO Andy Jassy, who has been there from the beginning, explained how these core systems developed out of need over a three-year period beginning in 2000, and, before they knew it, without any real planning, they had the makings of a business that would become AWS.
Creating internal systems
It began way back in the 2000 timeframe when the company wanted to launch an e-commerce service called Merchant.com to help third-party merchants like Target or Marks & Spencer build online shopping sites on top of Amazon’s e-commerce engine. It turned out to be a lot harder than they thought to build an external development platform, because, like many startups, when it launched in 1994, it didn’t really plan well for future requirements. Instead of an organized development environment, they had unknowingly created a jumbled mess. That made it a huge challenge to separate the various services to make a centralized development platform that would be useful for third parties.
So very quietly around 2000, we became a services company with really no fanfare.Andy Jassy, AWS CEO
At that point, the company took its first step toward building the AWS business by untangling that mess into a set of well-documented APIs. While it drove the smoother development of Merchant.com, it also served the internal developer audience well, too, and it set the stage for a much more organized and disciplined way of developing tools internally going forward.
“We expected all the teams internally from that point on to build in a decoupled, API-access fashion, and then all of the internal teams inside of Amazon expected to be able to consume their peer internal development team services in that way. So very quietly around 2000, we became a services company with really no fanfare,” Jassy said.
At about the same time, the company was growing quickly and hiring new software engineers, yet they were still finding, in spite of the additional people, they weren’t building applications any faster. When Jassy, who was Amazon CEO Jeff Bezos’ chief of staff at the time, dug into the problem, he found a running complaint. The executive team expected a project to take three months, but it was taking three months just to build the database, compute or storage component. Everyone was building their own resources for an individual project, with no thought to scale or reuse. (I think you can guess where this is going.)
The internal teams at Amazon required a set of common infrastructure services everyone could access without reinventing the wheel every time, and that’s precisely what Amazon set out to build — and that’s when they began to realize they might have something bigger.
A perfectly wonderful awful idea
Jassy tells of an executive retreat at Jeff Bezos’ house in 2003. It was there that the executive team conducted an exercise identifying the company’s core competencies — an exercise they expected to last 30 minutes, but ended up going on a fair bit longer. Of course, they knew they had skills to offer a broad selection of products, and they were good at fulfilling and shipping orders, but when they started to dig they realized they had these other skills they hadn’t considered.
In retrospect it seems fairly obvious, but at the time I don’t think we had ever really internalized that.Andy Jassy, AWS CEO
As the team worked, Jassy recalled, they realized they had also become quite good at running infrastructure services like compute, storage and database (due to those previously articulated internal requirements). What’s more, they had become highly skilled at running reliable, scalable, cost-effective data centers out of need. As a low-margin business like Amazon, they had to be as lean and efficient as possible.
It was at that point, without even fully articulating it, that they started to formulate the idea of what AWS could be, and they began to wonder if they had an additional business providing infrastructure services to developers.
“In retrospect it seems fairly obvious, but at the time I don’t think we had ever really internalized that,” Jassy explained.
The operating system for the internet
They didn’t exactly have an “aha” moment, but they did begin to build on the initial nugget of an idea that began at the retreat — and in the Summer of 2003, they started to think of this set of services as an operating system of sorts for the internet. Remember, this is still three years before they launched AWS, so it was an idea that would take time to bake.
I don’t think any of us had the audacity to predict it would grow as big or as fast as it has.Andy Jassy, AWS CEO
“If you believe companies will build applications from scratch on top of the infrastructure services if the right selection [of services] existed, and we believed they would if the right selection existed, then the operating system becomes the internet, which is really different from what had been the case for the [previous] 30 years,” Jassy said.
That led to a new discussion about the components of this operating system, and how Amazon could help build them. As they explored further, by the Fall of 2003 they concluded that this was a green field where all the components required to run the internet OS had yet to be built — at which point I’m imagining their eyes lit up.
“We realized we could contribute all of those key components of that internet operating system, and with that we went to pursue this much broader mission, which is AWS today, which is really to allow any organization or company or any developer to run their technology applications on top of our technology infrastructure platform.”
Then they set out to do just that — and the rest, as they say, is history. A few years later the company launched their Infrastructure as a Service (a term that probably didn’t exist until later). It took time for the idea to take hold, but today it’s a highly lucrative business.
AWS was first to market with a modern cloud infrastructure service when it launched Amazon Elastic Compute Cloud in August, 2006. Surprisingly, it took several years before a competitor responded. As such, they control a vast amount of market share, at least for now. Rest assured, some very well-heeled competitors like Microsoft, Google, IBM and others are gunning for them.
When asked if he ever foresaw the success they’ve achieved, Jassy was humble, saying, “I don’t think any of us had the audacity to predict it would grow as big or as fast as it has.”
But given how the company carefully laid the groundwork for what would become AWS, you have to think that they saw something here that nobody else did, an idea that they believed could be huge. As it turned out, what they saw was nothing less than the future of computing.
Company says president wanted to harm Bezos in awarding of cloud-computing deal to Microsoft
WASHINGTON— Amazon.com Inc. said President Trump exerted “improper pressure” on the Pentagon to keep a lucrative cloud-computing deal from going to his perceived enemy, company founder Jeffrey Bezos.
In a complaint filed in the U.S. Court of Federal Claims in Washington, Amazon said the president “launched repeated public and behind-the-scenes attacks” on the contract and the company to steer the contract away from Amazon and Mr. Bezos, according to the complaint, which was made public Monday.
Amazon was long considered the favorite to win the Joint Enterprise Defense Infrastructure, or JEDI, contract, which is valued at as much as $10 billion over the next decade. The company’s bid was clouded by conflict-of-interest allegations, however, which are still under investigation by the Pentagon’s inspector general.
While the Pentagon initially concluded that the alleged conflict didn’t affect the integrity of the procurement process, it nonetheless ruled in the end that Microsoft Corp. was more qualified for the job.
In its newly public bid protest, Amazon says that Pentagon officials made numerous missteps in evaluating its application because of pressure from Mr. Trump.
The president on July 19 called for an investigation of the Pentagon contract, before the award. “I’m getting tremendous complaints about the contract with the Pentagon and Amazon,” Mr. Trump told reporters at the time. “I will be asking them to look very closely to see what’s going on.” Mr. Trump also issued tweets in which he complained about the process.
Mr. Trump has blamed Mr. Bezos for unfavorable coverage of his administration in the Washington Post, which Mr. Bezos bought in 2013 for $250 million. The Post says its editorial decisions are independent.
In recent congressional testimony, a top Pentagon technology official, Dana Deasy, denied that Mr. Trump or the White House influenced the JEDI selection process.
But Amazon’s protest argues: “Rarely, if ever, has a President engaged in such a blatant and sustained effort to direct the outcome of a government procurement—let alone because of personal animus and political objectives.”
The White House referred questions to the Pentagon, which in a statement said the “selection decision was made by an expert team of career public servants and military officers from across the Department of Defense and in accordance with DOD’s normal source-selection process.”
“There were no external influences on the source selection decision,” the statement said. “The department is confident in the JEDI award and remains focused on getting this critical capability into the hands of our warfighters as quickly and efficiently as possible.”
Microsoft, which has intervened in the Amazon lawsuit to defend the award, didn’t immediately comment.
Among other evidence for Mr. Trump’s alleged bias, Amazon’s complaint cites a passage of a recent book by a former speechwriter for Jim Mattis, a former Trump defense secretary. Author Guy Snodgrass says in the book, “Holding the Line,” that Mr. Trump directed Mr. Mattis to “screw Amazon” out of the JEDI contract by blocking its chance to bid on the JEDI deal. “Mattis demurred,” he added.
Amazon also says the Defense Department took “numerous actions” during the evaluation process that disadvantaged Amazon.
For example, Amazon says that the Pentagon “at the eleventh hour” changed its interpretation of the bid criteria, rejecting Amazon’s plan to use existing data centers to help fulfill the contract requirements. That—along with other unfair actions by the Pentagon—resulted in increases in Amazon’s total evaluated price, the company said.
Athena is serverless. You can quickly query your data without having to setup and manage any servers or data warehouses. Just point to your data in Amazon S3, define the schema, and start querying using the built-in query editor. Amazon Athena allows you to tap into all your data in S3 without the need to set up complex processes to extract, transform, and load the data (ETL).
I am a cloud support engineer here at AWS, and customers often ask me how they can limit Amazon S3 bucket access to a specific AWS Identity and Access Management (IAM) role. In general, they attempt to do this the same way that they would with an IAM user: use a bucket policy to explicitly Deny all Principals (users and roles) to which they do not want to grant access. The drawback with this approach is the required maintenance of the bucket policy. If a new IAM user were added to the account with “s3:*” for the Action, the user would be granted access to the bucket. Rather than specify the list of users whose access you want to block, you can invert the logic and leverage the NotPrincipalelement in the bucket policy’s Deny statement. This element creates an explicit Deny for any user that is not listed in its value.
However, this inverted logic approach proves problematic with IAM roles because the role’s Principal value is composed of two Amazon Resource Names (ARNs), the role ARN and the assumed-role ARN. The role ARN is the identifier for the IAM role itself and the assumed-role ARN is what identifies the role session in the logs. When using the NotPrincipal element, you must include both ARNs for this approach to work, and the second of these ARNs should include a variable name. Normally you would specify a wildcard where the variable string would go, but this is not allowed in a Principal or NotPrincipal element. In this blog post, I show how you can restrict S3 bucket access to a specific IAM role or user within an account using Conditions instead of with the NotPrincipal element. Even if another user in the same account has an Admin policy or a policy with s3:*, they will be denied if they are not explicity listed. You can use this approach, for example, to configure a bucket for access by instances within an Auto Scaling group. You can also use this approach to limit access to a bucket with a high-level security need.
