Earlier this year, engineers at Wal-Mart Stores Inc. who track rivals’ prices online got a rude surprise: the technology they were using to check Amazon.com several million times a day suddenly stopped working.

Losing access to Amazon.com Inc.’s data was no small matter. Like most big retailers, Wal-Mart relies on computer programs that scan prices on competitors’ websites so it can adjust its listings accordingly. A difference of even 50 cents can mean losing a sale.

But a new tactic by Amazon to block these programs — known commonly as robots or bots — thwarted the Bentonville, Arkansas-based retailer.

Its technology unit, @WalmartLabs, was unable to work around the blockade for weeks, forcing it to retrieve Amazon’s data through a secondary source, according to a person familiar with the matter who was not authorized to speak publicly.

The previously unreported incident offers a case study in how Amazon’s technological prowess is helping it dominate the retail competition.

Now the largest online retailer in the world, Amazon is best known by consumers for its fast delivery, huge product catalog and ambitious moves into areas like original TV programming. But its mastery of the complex, behind-the-scenes technologies that power modern e-commerce is just as important to its success.

Dexterity with bots allows Amazon not only to see what its rivals are doing, but increasingly to keep them in the dark when it undercuts them on price or is quietly charging more.

“Benchmarking against Amazon is going to become hard,” said Guru Hariharan, a former Amazon manager who now sells pricing software to retailers as chief executive of Mountain View, California-based Boomerang Commerce.

A Wal-Mart spokesman declined to discuss the January episode but said the company improves its technology regularly and has multiple tools for tracking items. He said the company offers value not only through pricing but from discounts for in-store pickup and other benefits.

A spokeswoman for Amazon said the company is aware of competitors using bots to check its listings and denied any “campaign” to stop them. “Nothing has changed recently in how we manage bots on our site,” she said. Still, she said, “we prioritize humans over bots as needed.”

Bots can slow down a website, a big motivator for retailers to block them.

Reuters interviewed 21 people familiar with bots and how they are deployed, including current and former Wal-Mart employees, former Amazon employees and outside specialists. Many spoke only on condition of anonymity because they were not authorized to discuss the issues publicly.

Most pointed to Amazon’s leadership in the burgeoning bot wars.

The company’s technological edge has been good for its profit margin, and it’s proving a winning formula for investors.

Shares of the internet powerhouse have risen about 15-fold since the market’s bottom in March 2009, while the S&P 500 has more than tripled in value. Amazon hit $100 billion in annual sales in 2015 — faster than any company in history, it said.

Brave new world

Bot-driven pricing has represented a massive change for the retail industry since Amazon helped pioneer the practice more than a decade ago.

Traditionally, brick-and-mortar stores changed prices no more than weekly because of the time and expense needed to swap labels by hand.

In the world of e-commerce, though, retailers update prices with ease, sometimes multiple times a day, helped by algorithms that consider inventory levels, sales forecasts and rivals’ pricing data.

To stay in the game, companies such as online wholesaler Boxed, based in New York, depend on a variety of methods including bots to ensure they do not lag others’ price moves for even 20 minutes.

“That’s like a lifetime during Christmas,” said Chief Executive Chieh Huang, whose company sells bulk staples like toilet paper and pet food. “If we’re not decently priced, we’ll see it almost immediately” in sales declines.

Disguised as humans

Using bots to view massive amounts of data on public websites — a process known as crawling or scraping — has many purposes. Alphabet Inc.’s Google, for example, constantly crawls the Web to gather information for its search engine results and to sell ads.

In e-commerce, though, the use of bots has developed into a cat-and-mouse game. Companies try to thwart the practice on their own websites while aiming to penetrate their competitors’ defenses. Third-party services abound to help less-savvy retailers.

To protect data from rivals, some retail websites use what’s known as a “CAPTCHA” — typically a distorted string of letters and numbers that humans can read but most bots can’t. Amazon shies away from the practice because it annoys some customers.

For merchants seeking to evade such defenses, disguising their computer programs as real shoppers is key. Some pricing technology experts have programmed computer cursors to meander through a Web page in the way a person might, instead of going directly to the prized data. Another technique is to use multiple computer addresses so that retailers cannot track a barrage of clicks to a single source.

“It is an arms race,” said Keith Anderson, a senior vice president at e-commerce analytics firm Profitero, based in Ireland. “Every week or every month, there’s some new approach from both sides.”

Amazon’s maneuver that halted Wal-Mart in January took aim at a specialized Web browser called PhantomJS. Unlike, say, Internet Explorer, this browser is designed specifically for programmers — a telltale clue that its users are not typical shoppers. Amazon put up a digital curtain to hide its listings from PhantomJS users, according to three people familiar with the situation.

It was unclear how the move, which was not aimed at Wal-Mart in particular, affected other companies.

Tests conducted in recent weeks for Reuters show that among major U.S. retail chains, Amazon had by far the most sophisticated bot detection in place, both for its home page and for two popular items selected by Reuters because they change price frequently — a De’Longhi coffee maker and a Logitech webcam.

The tests were run by San Francisco-based Distil Networks, which sells anti-bot tools. In one of the tests, Distil programmed bots to hit each retailer’s website 3,000 times, but slowly enough to mimic a person clicking through listings. This tricked most retail behemoths, but not Amazon.

Blocked bots would not have seen, for instance, that Amazon’s price for the De’Longhi espresso machine changed four times in a single 24-hour period starting on the morning of April 25, according to price tracking website camelcamelcamel.com. During that time, the price swung by more than 10 percent, from a low of $80.06 to $88.16.

Swarming with bots

Despite Amazon’s capabilities, the sheer volume of crawling on its site is staggering. At times, as many as 80 percent of the clicks on Amazon product listings have been from bots, people familiar with the matter say, compared with just a third or more of the traffic on other large sites.

In addition to rivals seeking price data, that traffic includes bots from university researchers studying competition, search engines, advertising services and even fraudsters trying to break into Amazon accounts.

For Wal-Mart, a small group in Silicon Valley directs its automated pricing strategy while dozens of engineers in India and around the world handle the code, current and former Wal-Mart employees said.

Amazon had about 40 engineers who would covertly extract and organize rivals’ data with bots as of several years ago, one of the people interviewed said. Amazon did not discuss the size or structure of its teams working with bots.

According to one U.S. patent application, Amazon is working on encryption technology that would force bots, but not humans, to solve a complicated algorithm to gain access to its Web pages.

“Amazon has both the competency to detect bot traffic and the wherewithal to do something about it,” said Scott Jacobson, a former Amazon manager and now managing director of Madrona Venture Group. That “isn’t the case for most retailers.”

 
…

Microsoft Corp. on Wednesday turned up the heat on other technology giants by launching new image and video recognition products which could help it court businesses worried about running ads next to offensive content.

The Redmond, Washington-based company said its new Video Indexer can identify faces, voices and emotions in moving pictures. Separately, its Custom Vision Search lets companies build apps that recognize images with just a few lines of code.

For brands, knowing what’s in the videos that they sponsor has become a hot-button issue since major companies began canceling ad deals with Alphabet Inc’s Google this year over hate speech playing on its subsidiary YouTube.

Microsoft’s Video Indexer has similarities to a tool Google launched in March; Amazon.com Inc also said last month it could flag insulting images via a cloud-based service.

Microsoft’s latest moves underscore how its focus has evolved from its staple Windows software to the cloud, where it is competing with Amazon to sell data storage and computing power. Extra analytics such as image recognition may prove key to luring Web developers.

“It’s hard to understand what’s in the video” the longer it is, said Irving Kwong, a senior product director at Microsoft, in an interview ahead of the company’s developer conference Build. He said Video Indexer, which analyzes videos far faster than humans can, could help a user “harness and get more out of the video content that you have.”

The tools launched in preview by the Microsoft Cognitive Services unit on Wednesday, including a decision recommendation service, have one aim apart from winning business: data.

Microsoft views the tools as a way to put powerful computing into people’s hands and improve the tools at the same time, because processing more data is key to reaching artificial intelligence. Others including Amazon are pursuing this strategy, with the prize being a new revenue stream.

Research firm International Data Corporation has forecast the market for such tools will balloon to over $47 billion in sales in 2020 from $8 billion in 2016.

Microsoft pulled back the curtain on experiments that are further afield, too. It announced a new Cognitive Labs unit and the so-called Project Prague: technology to allow people to control computers simply with hand gestures.
…

Facebook is planning to intensify its crackdown on so-called clickbait websites, saying it will begin giving lower prominence to links that lead to pages full of deceptive or annoying advertisements.

The downgrade of the links was expected to take effect beginning on Wednesday on News Feed, the home page of Facebook where people go to see posts from friends and family.

Facebook said it wanted to downplay links that people post to websites that have a disproportionate volume of ads relative to content, or that have deceptive or sexually suggestive ads along the lines of “5 Tips to be Amazing in Bed” or “1 Crazy Tip to Lose Weight Overnight!”

Links to websites with pop-up ads or full-screen ads also would be downplayed, it said.

People scrolling through their News Feed are often disappointed when they click on such links and do not find valuable information, Andrew Bosworth, Facebook’s vice president of ads and business platform, said in an interview.

“People don’t want to see this stuff,” he said. “We’re just trying to figure out how to find it and rank it further down News Feed when possible.”

Facebook uses a computer algorithm to determine which posts people see first from friends and family, and it frequently refines the algorithm to keep up with spam or other concerns.

The company said in August it was adjusting the algorithm to downplay news stories with clickbait-style headlines, a style of headline that intentionally withholds information or misleads people to get them to click on them.

In December, facing criticism that hoaxes and fake news stories spread too easily on Facebook in the run-up to the U.S. presidential election on November 8, the company made it easier for people to report those kinds of posts.

Facebook, the world’s largest social media network with 1.9 billion monthly users, has enormous power with its algorithms to potentially drive traffic to media publishers or stymie it.

The company said it reviewed hundreds of thousands of websites linked to from Facebook to identify those with little substance but lots of disruptive or shocking ads.

Bosworth declined to name any websites Facebook wants to target. He said only publishers of spam needed to worry about seeing less traffic, and other publishers could see their traffic go up.

“This is a small number of the worst of the worst,” he said.
…

Drone-maker AeroVironment Inc. unveiled a small four-rotor surveillance helicopter on Tuesday that can be carried in a small pouch and launched from the palm of a hand.

The smaller size and simplicity of operation means it can used by ordinary soldiers, offering squads and other small military units the kind of surveillance capacity previously reserved for larger military units, where drones are operated by specialists.

AeroVironment said it delivered 20 of the 5-ounce (140-gram) Snipe unmanned aircraft to its first U.S. government client in April. The company declined to identify the government agency that purchased the drones, but Aviation Week reported last year that AeroVironment was developing prototypes for the U.S. Army.

Designed to worn as part of uniform

AeroVironment said the drone benefited from advances in technology achieved in the development of its Nano Hummingbird drone for DARPA, the Defense Advanced Research Projects Agency, which has been responsible for many technological and scientific breakthroughs used by the military.

Kirk Flittie, AeroVironment’s vice president in charge of unmanned aircraft systems, said in a statement the Snipe copter drone is “designed to be worn by its operator so it can be deployed in less than a minute.”

Battery life is 15 minutes

The aircraft, which is intended for intelligence and reconnaissance missions, can relay high-resolution images and record video both day and night. It can fly at speeds of 20 mph (35 kph), has a range of more than a kilometer (half-mile), and can fly for about 15 minutes on batteries, the statement said.

AeroVironment’s hand-launched Raven unmanned aircraft, which weighs 4.2 pounds (2 kg) and has a wingspan of 4.5 feet (1.4 meters), is one of the most widely used military surveillance drones, with more than 19,000 built.

Shares of AeroVironment dropped 0.2 percent to $29.13 within its 52-week range of $22.16 to $32.44.
…

When Moumouni Abdoulaye and his fellow herders in western Niger used to set off on scouting missions in search of water, they feared for their livestock – and for their own lives.

Unable to rely anymore on their traditional methods of predicting the weather amid increasingly erratic droughts and floods, and lacking modern climate information, they struggled to predict where, and when, they might find water in the vast arid region.

“We were living in limbo. Without knowledge, we constantly risked our lives,” said Abdoulaye, seeking shade under a tree from the fierce midday sun in Niger’s Tillabery region.

But a project to involve the region’s semi-nomadic people in the production of locally-specific, real-time weather forecasts – and provide them with radios and mobile phones to receive and share the information – is transforming the lives of tens of thousands of Nigeriens like Abdoulaye.

“Now we receive daily updates about rainfall, can call other communities to ask if they have had rain, and plan our movements accordingly,” Abdoulaye told the Thomson Reuters Foundation.

In Niger, as across much of Africa’s Sahel region, frequent droughts have impoverished many people and made it much harder to make a living from agriculture. That is happening in a West African country already consistently ranked at the bottom of the U.N. Human Development Index.

With climate change now exacerbating pressures, experts say there is a growing and urgent need for better climate information, to ensure farmers and pastoralists are equipped to cope with unpredictable rainfall and climate shocks.

Across Africa, only limited climate data is collected and made available, and information services are often not well understood, user-friendly, or followed up to help people put the information to use in adapting to climate threats, experts say.

Ensuring that communities play a role – alongside state and aid agencies – in generating and sharing weather information is the best way to get them to use it and to build their resilience to the growing pressures, said Blane Harvey of the Overseas Development Institute (ODI).

“Co-participation is very powerful because people will buy into a service if they’ve had a hand in producing it,” he said.

“Crucially, they bring in their local knowledge, which helps to downscale and triangulate more regionalized forecasts,” added Harvey, a research associate at the London-based think tank.

Collaboration crucial

A lack of weather stations across Africa means that forecasts, produced by national meteorological agencies, tend to be too broad to be of much use at a local level.

But a project launched in 2015, funded by the U.K. Department for International Development (DFID) and led by CARE International, is trying to improve the quality of and access to climate data for farmers and pastoralists in western Niger.

CARE’s project under the Building Resilience and Adaptation to Climate Extremes and Disasters (BRACED) program aims to help 450,000 people become better prepared for climate shocks, including through giving them access to better forecasts.

The goal is to help them diversify their farming and find ways of making money which are not so heavily impacted by climate change, in order to better withstand climate pressures.

For farmer Adamou Soumana, improved access to climate information has given his village a better understanding of the weather shocks they are encountering, and the confidence to adopt resilience boosting strategies such as using climate-adapted seeds, finding sustainable ways to harvest forest products, and storing harvests.

“Previously, if it rained in January, we rushed to plant our crops thinking the rainy season starts – when in fact it never comes before May,” he said.

“Now we understand climate shocks, and can plan our activities in advance. We feel more resilient,” he said.

The BRACED project has helped communities by acting as a broker between them and meteorological agencies, and ensuring agency partners are trained to interpret climate data, translate it into local languages and help people to make sense of the forecasts.

The project also connects local people who collect rainfall data, as well as other farming and pastoralist leaders, with community radio stations to share real-time information daily.

Incorporating traditional observations – such as when trees bloom or the way birds behave – and having regular discussions with communities is key to building and maintaining trust in climate information services, said Richard Ewbank of Christian Aid, another charity working on climate resilience issues.

“Having experts and community leaders together and combining local knowledge with scientific forecasts is the best way to agree on a climate scenario, and make key decisions for the coming season,” said the global climate advisor for the charity.

Life or death decisions

In addition to improving the quality of climate information and making it more relevant on a community-by-community basis, the BRACED project in Niger has provided mobile phones and radios to boost the spread of the forecasts.

“Receiving and sharing the information in this way not only helps pastoralists know when and where to move, it also builds relationships and trust between people,” said Amadou Adamou of the Association for the Revitalization of Livestock Breeding.

Good information can not only help pastoralists find water sources but also help them know when to sell their animals, especially if drought is on the way, according to Adamou.

The mobile phones and radios used are powered by solar cells, enabling pastoralists to get forecasts while on the move. They also are given to both male and female community chiefs to ensure women have equal access to the information.

While better climate data has improved resilience for many in Tillabery region, in both settled and nomadic communities, there is still much room for improvement, several experts said.

Residents want to see more meteorological advisers based locally who can help them have regular discussions about the forecasts.

They also want more help to convert the data into action on the ground such as diversifying the crops they grow and better planning the timing and direction of their migration routes in search of water. They also want the information service expanded to cover neighboring countries.

“Getting better forecasts is one thing. But having good, solid advice about what the information means, and discussions on how to use it to become more resilient, is what people in the region really want,” said Harouna Hama Hama of CARE.

For roaming communities like Abdoulaye’s – people who cross into neighboring Benin, Burkina Faso and Togo with their livestock – expanding the climate data effort to produce region-wide forecasts could mean the difference between life and death for many of their members, Abdoulaye said.

“Whenever some of our people head to these countries, they and the animals risk dying of thirst,” he said. “With better forecasts, and for the whole region, we could lose fewer lives.”
…

Amazon is giving its voice-enabled Echo speaker a touch screen and video-calling capabilities as it competes with Google’s efforts at bringing “smarts” to the home.

The new device, called Echo Show, goes on sale on June 28 for $230.

The market for voice-assisted speakers is small but growing. Research firm eMarketer expects usage of the speakers to more than double, with nearly 36 million Americans using such a device at least once a month by year’s end.

Amazon’s Echo is expected to continue its dominance, with a share of nearly 71 percent, though eMarketer expects Google’s Home speaker to cut into that share in the coming years.

Amazon says it’s also bringing calling and messaging features to its existing Echo and Echo Dot devices and the Alexa app for phones.
…

The website for the Federal Communications Commission has come under attack.

Initially, the problems were believed to have been caused by comedian John Oliver, who on Sunday urged his viewers to leave comments on the site about the FCC’s plans to revisit net neutrality rules.

Net neutrality rules were implemented in 2015 and required internet service providers to treat all traffic equally. New FCC chairman Ajit Pai has said he will review the rules, arguing they are “holding back investment, innovation and job creation.”

The FCC, which “regulates interstate and international communications by radio, television, wire, satellite, and cable,” says the website attacks were coordinated, distributed denial of service attacks, not a surge in traffic.

“These actors were not attempting to file comments themselves, rather they made it difficult for legitimate commenters to access and file with the FCC,” chief information officer David Bray said. “While the comment system remained up and running the entire time, these distributed denial of service events tied up the servers and prevented them from responding to people attempting to submit comments.”

On his show, “Last Week Tonight,” Oliver said, “Every internet group needs to come together … gamers, YouTube celebrities, Instagram models, Tom from MySpace if you’re still alive. We need all of you,” he said.

The FCC will vote on net neutrality rules on May 18.
…

Exploration of Mars has not proceeded without setbacks, but that did not discourage scientists trying to find the answer to one of the crucial questions – has the red planet ever sustained life? If the answer is positive, it would mean that we are not alone in the universe. Scientists at the European Space Agency ESA have already moved on from last year’s crash of their lander, preparing its orbiting parent spacecraft to start looking for life-related gases. VOA’s George Putic reports.
…

Facebook must remove postings deemed as hate speech, an Austrian court has ruled, in a legal victory for campaigners who want to force social media companies to combat online “trolling.”

The case — brought by Austria’s Green party over insults to its leader — has international ramifications as the court ruled the postings must be deleted across the platform and not just in Austria, a point that had been left open in an initial ruling.

The case comes as legislators around Europe are considering ways of forcing Facebook, Google, Twitter and others to rapidly remove hate speech or incitement to violence.

Germany’s cabinet approved a plan last month to fine social networks up to 50 million euros ($55 million) if they fail to remove such postings quickly and the European Union is considering new EU-wide rules.

Facebook and its lawyers in Vienna declined to comment on the ruling, which was distributed by the Greens and confirmed by a court spokesman.

Court asks about automation

Strengthening the earlier ruling, the Viennese appeals court ruled on Friday that Facebook must remove the postings against Greens leader Eva Glawischnig as well as any verbatim repostings, and said merely blocking them in Austria without deleting them for users abroad was not sufficient.

The court added it was easy for Facebook to automate this process. It said, however, that Facebook could not be expected to trawl through content to find posts that are similar, rather than identical, to ones already identified as hate speech.

The Greens hope to get the ruling strengthened further at Austria’s highest court. They want the court to demand Facebook remove similar — not only identical — postings, and to make it identify holders of fake accounts.

Greens to seek damages

The Greens also want Facebook to pay damages, which would make it easier for individuals in similar cases to take the financial risk of taking legal action.

“Facebook must put up with the accusation that it is the world’s biggest platform for hate and that it is doing nothing against this,” said Green parliamentarian Dieter Brosz.

Chief Executive Mark Zuckerberg has said hate speech has no place on the platform and the company has published a policy paper on how it wants to work against false news.

 
…

Facebook says it has deleted tens of thousands of accounts in Britain ahead of the June 8 general election in a drive to battle fake news.

The tech giant also took out newspaper advertisements in Britain’s media offering advice on how to spot such stories. The ads suggest that readers should be “skeptical of headlines,” and “look closely at the URL.”

The company says it has made improvements to help it detect fake news accounts more effectively.

Simon Milner, the tech firm’s U.K. director of policy, says the platform wants to get to the “root of the problem” and is working with outside organizations to fact check and analyze content around the election.

Milner says Facebook is “doing everything we can to tackle the problem of false news.”
…

China will further tighten its internet regulations with a pledge Sunday to strengthen controls over search engines and online news portals, the latest step in President Xi Jinping’s push to maintain strict Communist Party control over content.

Xi has made China’s “cyber sovereignty” a top priority in his sweeping campaign to bolster security. He has also reasserted the ruling Communist Party’s role in limiting and guiding online discussion.

The five-year cultural development and reform plan released by the party and State Council, or Cabinet, calls for a perfecting of laws and rules related to the internet.

Qualifications for online reporters

That includes a qualification system for people working in online news, according to the plan, carried by the official Xinhua news agency.

“Strike hard against online rumors, harmful information, fake news, news extortion, fake media and fake reporters,” it said, without giving details.

Xi has been explicit that media must follow the party line, uphold the correct guidance on public opinion and promote “positive propaganda.”

The plan comes on top of existing tight internet controls, which includes the blocking of popular foreign websites such as Google and Facebook.

Security threat cited

The government last week issued tighter rules for online news portals and network providers. Regulators say such controls are necessary in the face of growing security threats, and are done in accordance with the law.

Speaking more broadly about the country’s cultural sector, the plan calls for efforts to reinforce and improve “positive propaganda.” The plan also calls for more effort to be put into promoting China’s point of view and cultural soft power globally, though without giving details. 

 
…

3D printing with metals is rapidly changing the way parts are being manufactured because it is now possible to create continuous complex shapes. Where once parts had to be welded to close the gaps, they can now be made as one solid piece. NASA’s scientists say they can now print flexible material made of intertwined metal rings. VOA’s George Putic reports.
…