Categories
knowledge connexions

Open source AI chips making Green Waves: Bringing energy efficiency to IoT architecture

What if machine learning applications on the edge were possible, pushing the limits of size and energy efficiency? GreenWaves is doing this, based on an open-source parallel ultra low power microprocessor architecture. Though it’s early days, implications for IoT architecture and energy efficiency could be dramatic.

The benefits open source offers in terms of innovation and adoption have earned it a place in enterprise software. We could even go as far as to say open source is becoming the norm in enterprise software. But open source hardware, chips to be specific, and AI chips to be even more specific? Is that a thing?

Also: AI Startup Gyrfalcon spins plethora of chips for machine learning 

Apparently it is. GreenWaves, a startup based in Grenoble, France, is doing just that. GreenWaves is developing custom, ultra-low power specialized chips for machine learning. These specialized chips leverage parallelism and a multi-core architecture to run machine learning workloads at the edge, on battery-powered devices with extreme limitations. The chips GreenWaves makes are based on open source designs, and are making waves indeed.

GreenWaves just announced a 7M€ Series A Funding with Huami, Soitec, and other investors. As per the announcement, funds will finance the sales ramp of GreenWaves’ first product, GAP8, and the development of the company’s next generation product. ZDNet discussed with Martin Croome, GreenWaves VP of Product Development, to find out what this is all about.

Open source microprocessors for IoT

First off, what does open source even mean when we’re talking about microprocessors? As Croome explained, what is open source in this case is the instruction set architecture (ISA), and the parallel ultra low power computing platform (PULP) that sits on top of it.

Also: AI Startup Cornami reveals details of neural net chip

GreenWaves is a fabless chip maker. What this means is that it designs chip architectures, and it then builds them by outsourcing to some hardware manufacturer. So, GreenWaves uses low-level building blocks, customizing them and combining them with its own extensions, to produce a proprietary design.

This approach is somewhat reminiscent of open core software: An open source core, with custom extensions that add value. The building blocks that GreenWaves is using is the RISC-V instruction set, and PULP.

PULP is an open source parallel ultra-low power platform on which innovative chip designs can be created.

RISC-V is an open-source hardware ISA based on established reduced instruction set computer (RISC) principles. RISC-V can be used royalty-free for any purpose and began in 2010 at the University of California, Berkeley. RISC-V has many contributors and users in the industry. As Loic Lietar, GreenWaves’ co-founder and CEO noted, the likes of Nvidia and Google also use RISC-V. This means RISC-V contributions grow, and anyone can benefit.

Also: AI chips for big data and machine learning: GPUs, FPGAs

PULP is a parallel ultra-low power multi-core platform aiming to satisfy the computational demands of IoT applications requiring flexible processing of data streams generated by multiple sensors. PULP wants to meet the computational requirements of IoT applications, without exceeding the power envelope of a few milliwatts typical of miniaturized, battery-powered systems.

PULP started as a joint effort between ETH Zürich and the University of Bologna in 2013. GreenWaves sprung out of PULP in 2014, as its CTO and co-founder, Eric Flamand, was also a co-founder in PULP. Fast forward to today, and GreenWaves has 20 employees, shipped a first batch of GAP8 chips to clients, and raised a total of €11.5.

Real-time processing at the edge

Croome noted that GreenWaves needed much less capital than what chip startups usually need, which is mostly spent in getting IP rights for designs. GreenWaves did not have to do this, and this made financing easier. Or, as Lietar put it, a few years ago, when GreenWaves would mention open source chips, there was a good chance they would be thrown out of the room. Not anymore.

Also: AI startup Flex Logix touts vastly higher performance than Nvidia

So, what’s special about GAP8, what can it be used for, and how?

GAP8 has an integrated, hierarchical architecture. It hosts 8 extended RISC-V cores and a HWCE (Hardware Convolution Engine). GreenWaves promises ultra low power 20x better than the state-of-the-art on art on content understanding. Content, in this context, can mean anything from image to sound or vibration sensor input.

What GAP8 is designed to do is to process that data at the edge in real time, bypassing the need to collect and send for processing to some remote data center. In order to do this, it has to be fully programmable, agile, and have low installation and operation cost.

gap8.jpg

GAP8’s architecture

The agile part is there, as GAP8 can wake up from a sleep state in 0.5 milliseconds. As Croome noted, such chips deployed at the edge spend a big part of their lifetime actually doing nothing. So it was important to design something that sleep consuming as little power as possible, and then wake up, and switch modes of operation, as quickly and efficiently as possible.

Also: AI’s insatiable appetite for silicon requires new chips

The low installation and operation cost is there, too, as GreenWaves promises years of operation on batteries, or even better solar cells. As GAP8 can operate over wireless solutions such as LoRa, GreenWaves also promises a 10- to 100- fold cost reduction over wired installations.

So, what can GAP8 do? Clients are using GAP8 for things such as counting people or objects, vibration and sound analysis, object recognition, and more. Some areas of application are smart cities, industry, security, and consumer applications. The really interesting part, however, is in how it all works.

Deploying machine learning models

All these applications are based on using machine learning, and more specifically, neural networks. GAP8 takes care of the inference, which means the models have to be trained first, and then deployed on GAP8. And this is where it gets a bit tricky.

Also: Chip startup Efinix hopes to bootstrap AI efforts in IoT

GAP8 is programmable via C or C++. So how does one get from a model built using TensorFlow, or PyTorch, and other machine learning libraries, to deployment on a GAP8? The software stack for this is open source and available on GitHub.

Examples exist for the development flow from TensorFlow to C. However, there’s a couple of gotchas. First, currently GAP8 only works with TensorFlow. Croome said this a matter of resources and priorities, and integration with other frameworks will be provided as well. For the time being, he added, what people do is to port models created in other frameworks to TensorFlow via ONNX.

Then, if you’re expecting a one-click deployment, you’re in for a disappointment. As Croome explained, the flow is tools based rather than being monolithic. This means that a number of tools provided by GreenWaves have to be utilized in order to deploy models to GAP8.

Croome noted that “all the functionality of GAP8 is visible to the programmer but that we do provide pre-written and optimized code as a ‘starting block’ for getting something up and running quickly. The HWCE accelerates the convolution operation however like all hardware blocks it works on specific convolution types. If it doesn’t match a specific layer then this can always be accelerated on the cluster cores programatically.”

Bringing energy efficiency to IoT architecture

The important thing here, however, is the ability to effectively process data at the edge. With a processor like GAP8, Croome noted, one can analyze the content produced by a rich data sensor and only upload the outcome, for example how many people are in a room:

Also: Meet Jetson Xavier: Nvidia’s AI chip

“This may well be uploaded into a time series database via an IoT Application platform (which may also only be hit after transmission over a low-speed LPWAN type network further minimizing data transfer). The energy spent in doing this analysis and the wireless transmission of the results, which can be seen as an ultimate compression, is far less than the wireless transmission of the raw data.”

greenwaveapps.jpg

Some of the applications low-power AI chips like GAP8 can be used for, simplifying IoT architecture

Although we have seen things such as deploying Hadoop at the edge, this would probably make little sense here. AI algorithms that operate on aggregate data from multiple sensors or access very large databases on the pre-compressed data are clearly better run on generic platforms on the edge (as opposed to very edge) or in the cloud, according to Croome.

“For a one in many face recognition application, the extraction of the key features would be run on GAP8 in the sensing device, the result would be uploaded and the matching would run in the cloud. This would be the best balance from a system point of view, for power consumption and from a SW engineering perspective,” Croome said.

Lietar said GreenWaves has been one step ahead of the market in identifying and serving this segment that is now widely recognized. Croome noted the state of the art in machine learning is evolving rapidly. He went on to add, however, that because GAP8 is not specialized, it can adapt well to new topologies and operators while retaining a best in class energy efficiency.

Innovation that leads to optimized energy efficiency and can simplify technical architecture – what’s not to like?

Content retrieved from: https://www.zdnet.com/article/open-source-ai-chips-making-green-waves-bringing-energy-efficiency-to-iot-architecture/.

Categories
knowledge connexions

From big data to AI: Where are we now, and what is the road forward?

It took AI just a couple of years to go from undercurrent to mainstream. But despite rapid progress on many fronts, AI still is something few understand and fewer yet can master. Here are some pointers on how to make it work for you, regardless of where you are in your AI journey.

In 2016, the AI hype was just beginning, and many people were still cautious when mentioning the term “AI”. After all, many of us have been indoctrinated for years to avoid this term, as something that had spread confusion, over-promised, and under-delivered. As it turned out, the path from big data and analytics to AI is a natural one.

Not just because it helps people relate and adjust their mental models, or because big data and analytics were enjoying the kind of hype AI has now, before they were overshadowed by AI. But mostly because it takes data — big or not-so-big — to build AI.

It also takes some other key ingredients. So, let’s revisit Big Data Spain (BDS), one of the biggest and most forward-thinking events in Europe, which marked the transition from big data to AI a couple of years back, and try to answer some questions on AI based on what we got from its stellar lineup and lively crowd last week.

Can you fake it till you make it?

Short answer: No, not really. One of the points in that Gartner analytics maturity model was that if you want to build AI capabilities (the predictive and prescriptive end of the spectrum), you have to do it on a solid big data foundation (the descriptive and diagnostic end of the spectrum).

Part of that is all about the ability to store and process massive amounts of data, but that really is just the tip of the iceberg. Technical solutions for this are in abundance these days, but as fellow ZDNet contributor Tony Baer put it, to build AI, you should not forget about people and processes.

More concretely: Don’t forget about data literacy and data governance in your organization. It has been pointed out time and again, but these really are table stakes. So, if you think you can develop AI solutions in your organization by somehow leapfrogging the evolutionary chain of analytics, better think again.

analytic-maturity.jpg

Gartner’s analytics maturity model may be a good starting point to explain and prepare the transition to AI. (Image: Gartner)

As Oscar Mendez, Stratio CEO, emphasized in his keynote, to go beyond flashy AI with often poor underpinnings, a holistic approach is needed. Getting your data infrastructure and governance right, and finding and training the right machine learning (ML) models on this can yield impressive results. But there is a limit to how far these can take you, amply demonstrated by everyday fails by the likes of Alexa, Cortana, and Siri.

The key message here is that bringing context and reasoning capabilities in play is needed to more closely emulate human intelligence. Mendez is not alone in this, as this is something shared by AI researchers such as Yoshua Bengio, one of Deep Learning’s top minds. Deep Learning (DL) excels in pattern matching, and the data and compute explosion can make it outperform humans in tasks based on pattern matching.

Intelligence, however, is not all about pattern matching. Reasoning capabilities cannot be built on ML approaches alone — at least not for the time being. So what is needed is a way to integrate less hyped AI approaches, of the so-called symbolic line: Knowledge representation and reasoning, ontologies, and the like. This is a message we have been advocating, and to see it take center-stage in BDS was an affirmation.

Should you outsource it?

Short answer: Perhaps, but you should be very considerate about it. So, let’s not beat around the bush: AI is hard. Yes, you should definitely build on foundational capabilities such as data governance, because this is good for your organization anyway. Some, like Telefonica, have managed to get from Big Data to AI, by executing strategic initiatives. But it’s no easy feat.

This point has been validated by what is probably the most reliable survey for ML adoption, answered by more than 11K respondents. Paco Nathan from Derwen presented results and insights from an O’Reilly survey he has instrumented, which more or less confirmed what we knew: There is a growing gap between the AI haves and have-nots.

On the one end of the spectrum, we have the Googles and Microsofts of the world: Organizations applying AI as a core element of their strategy and operation. Their resources, data, and know-how are such that they are leading the AI race. And then there are also adopters, working on applying AI in their domains, and laggards, buried too deep in technical debt to be able to do anything meaningful in terms of AI adoption.

p0043.jpg Leaders, adopters, laggards — the machine learning version. (Image: Paco Nathan / Derwen)

AI leaders have offerings that, on the face of it, seem to “democratize” AI. Both Google and Microsoft presented those in BDS, showcasing, for example, demos in which an image recognition application was built in a point and click fashion in a few minutes.

The message was clear: Let us worry about models and training, and you focus on the specifics of your domain. We can identify mechanical parts, for example — just feed us with your specific mechanical parts, and you are good to go.

Google also announced some new offerings in BDS: Kubeflow and AI Hub. The idea behind them is to orchestrate ML pipelines similarly to what Kubernetes does for Docker containers for applications, and to become a Github for ML models, respectively. These are not the only offerings that promise similar advantages. They sound alluring, but should you use them?

Who would not want to jump the AI queue, and get results here and now without all the hassle, right? This is indeed a pragmatic approach, and one that can get you ahead of the competition. There’s just one problem there: If you outsource your AI entirely, you are not going to develop the skills required to be self-sufficient in the mid-to-long term.

Think of Digital Transformation. Yes, going digital, exploring technologies and redesigning processes is hard. Not all organizations got it, or dedicated enough resources to it. But the ones that did are now ahead of the curve. AI has similar, if not greater, potential to disrupt and differentiate. So, while getting immediate results is great, investment in AI should still be seen as a strategic priority.

The one part you can be less skeptical about outsourcing is infrastructure. For most organizations, the numbers of maintaining your own infrastructure just don’t add up at this point. The economy of scale, head start, and peace of mind that running your infrastructure in the cloud can give are substantial benefits.

Where do we go from here?

Short answer: To the moon and back. It seems like the ML feedback loop is in full swing. So, while adopters are trying to keep up and laggards keep lagging, leaders are getting more and more advanced.

As pointed out in Google Partner Engineering Iberia / Italy Pablo Carrier’s presentation, compute is going to grow exponentially if you try to improve accuracy in DL linearly. In the past six years there was a 10-million fold increase in compute. That’s hard to keep up with even if you are Google Cloud, let alone if you are not.

A rising trend in DL is distribution. In an overview shown in another Google presentation, by Viacheslav Kovalevskyi, technical lead at Google Cloud AI, a word of warning was shared before embarking on the theory and practice of distributed DL: If possible, avoid it. If you really must do it, be aware there is an overhead associated with distribution, and be prepared to pay the price, both in terms of compute and complexity and in terms of footing bills.

Kovalevskyi offered a historical perspective on the different ways of using distributed DL — distributing the data, the model, or both. Distributing data is the easiest approach, distributing both is the hardest. But, in any case, distributed DL is “fairy tale zone” — you will not get a k-times increase in performance by increasing your compute times k.

Of course, Google’s presentation was focused on TensorFlow on Google Cloud, but this is not the only way to go. Databricks has just announced support for HorovodRunner to faciliate distributed DL using Horovod. Horovod is an open source framework, introduced by Uber, also utilized by Google. It’s not the only game in town though.

In a presentation given by Marck Vaisman, Microsoft data scientist and Azure data/AI technical specialist, alternatives were presented, using both Python and R without Spark in the mix. Dask, an open source library for Python, was highlighted. Dask promises advanced parallelism for analytics, working in tandem with projects like Numpy, Pandas, and Scikit-Learn.

And last but not least, graphs and graph databases were also a key theme throughout BDS: Microsoft’s knowledge graph, AWS Neptune, and Oracle Labs using graph analytics with Notebooks. As this is a topic we are following closely, we’ll be revisiting it shortly. Another Google insight to mention here: an internal analysis showed that most of Google’s ML models operate on structured data.

Cloud, distribution, and bringing structure to ML via graphs are some key themes to keep in mind for the future. We will continue to cover those as the journey from Big Data to AI progresses.

Content retrieved from: https://www.zdnet.com/article/from-big-data-to-ai-where-are-we-now-and-what-is-the-road-forward/.

Categories
knowledge connexions

Human in the loop: Machine learning and AI for the people

HITL is a mix and match approach that may help make ML both more efficient and approachable.

Paco Nathan is a unicorn. It’s a cliche, but gets the point across for someone who is equally versed in discussing AI with White House officials and Microsoft product managers, working on big data pipelines and organizing and part-taking in conferences such as Strata in his role as Director, Learning Group with O’Reilly Media.

Nathan has a mix of diverse background, hands-on involvement and broad vision that enables him to engage in all of those, having been active in AI, Data Science and Software Engineering for decades. The trigger for our discussion was his Human in the Loop (HITL) framework for machine learning (ML), presented in Strata EU.

HITL is a mix and match approach that may help make ML both more efficient and approchable. Nathan calls HITL a design pattern, and it combines technical approaches as well as management aspects.

HITL combines two common ML variants, supervised and unsupervised learning. In supervised learning, curated (labeled) datasets are used by ML experts to train algorithms by adjusting parameters, in order to make accurate predictions for incoming data. In unsupervised learning, the idea is that running lots of data through an algorithm will reveal some sort of structure.

The less common ML variant that HITL builds on is called semi-supervised, and an important special case of that is known as “active learning.” The idea is to take an ensemble of ML models, and let them “vote” on how to label each case of input data. When the models agree, their consensus gets used, typically as an automated approach.

When the models disagree or lack confidence, decision is delegated to human experts who handle the difficult edge cases. Choices made by experts are fed back to the system to iterate on training the ML models.

Nathan says active learning works well when you have have lots of inexpensive, unlabeled data — an abundance of data, where the cost of labeling itself is a major expense. This is a very common scenario for most organizations outside of the Big Tech circle, which is what makes it interesting.

But technology alone is not enough. What could be a realistic way to bring ML, AI, and automation to mid-market businesses?

AI for the people

In Nathan’s experience, most executives are struggling to grasp what the technology could do for them and identify suitable use cases. Especially for mid-market businesses, AI may seem like a far cry. But Nathan thinks they should start as soon as possible, and not look to outsource, for a number of reasons:

We are at a point where competition is heating up, and AI is key. Companies are happy to share code, but not data. The competition is going to be about data, who has the best data to use. If you’re still struggling to move data from one silo to another, it means you’re behind at least 2 or 3 years.

Better allocate resources now, because in 5 years there will already be the haves and have nots. The way most mid-market businesses get on board is by seeing, and sharing experiences with, early adopters in their industry. This gets them going, and they build confidence.

Getting your data management right is table stakes – you can’t talk about AI without this. Some people think they can just leapfrog to AI. I don’t think there will be a SaaS model for AI that does much beyond trivialize consumer use cases. “Alexa, book me a flight” is easy, but what about “Alexa, I want to learn about Kubernetes”? It will fall apart.

Not everything will be subscription based, and you have to consider that market leaders will not share the viable parts of their business. They may say so, but they don’t — there are many examples.

analytic-maturity.jpg

Gartner’s analytics maturity model may be a good starting point to explain and prepare the transition to AI. Image: Gartner

Nathan says Big Tech tends to automate use cases that are easy for them, which means they do not involve interaction with humans, because that’s not in their DNA. This is why he recommends that businesses work with second or third tier experts who are also aware of the business pain points.

And even if “outsourcing AI” was an option, it would still not be a great idea, for a number of reasons. Not just because of the bias in the datasets or the ethical reasons, but also because it is important to leverage knowledge within the organization:

“To get to artificial intelligence, you have to start with human intelligence. Knowledge does not come from IT, it comes from line of business and sales, and nobody outsources those.”

Leveraging uncertainty

Nathan says he has witnessed a similar situation in the early days of Data Science, and although he acknowledges there is some distance to cover, he believes executives will eventually get it:
Decision makers are used to making judgments. Any CEO understands statistics at a gut level, because that’s what they do every day. They may not know the math behind it, but the idea of collecting evidence, iterating on it and basing decisions on this is intuitive for executives.
Nathan says he believes in HITL because it’s based on two things.

First, it combines teams of machines and people. Currently, very few people in HR have even considered this, but Nathan thinks that going forward we’ll need things such as role descriptions for automation:

Many things we considered exclusively for people so far, will have to be considered for people and machines. If only for the sake of compliance, auditing, and continuity, these things have to be be taken very seriously.

In my experience all organizations have projects they do not embark upon because they do not have the resources. Once their processes are automated, people will be freed to embark on those.

expert.jpg

Leveraging human expertise alongside AI is a way to develop and use AI.

Which brings us to the second point about HITL. Nathan points out that we typically think of ML as a way to identify patterns and generalize using lots of data, but we may be about to witness a paradigm shift:

Rather than just recognize patterns, we can use ML to look at data and identify opportunities, by identifying uncertainty. This is not about risk — there’s no upside to risk, you just buy insurance. But if you can separate risk from uncertainty, you can profit, because that’s where the opportunities are.

With active learning, we have a way to identify where the uncertainties lie in our dataset. We can filter out the risk, and bring in the human experts to focus on the opportunity. We are starting to see companies that do just that, for example Stitch Fix.

I believe in this mixed model of augmenting experts. In any vertical, there is this proficiency threshold. You can achieve up to 80 percent proficiency, like someone who is good in that field. You can go up to say 95 percent, which is what experts achieve.

Beyond that, you get diminishing returns — chaos and churn, judgment calls and experts disagreeing with each other. These are areas of exploration, there are no perfectly right answers, but we can actually leverage that.

Content retrieved from: https://www.zdnet.com/article/human-in-the-loop-machine-learning-and-ai-for-the-people/.

Categories
knowledge connexions

Human in the loop: Machine learning and AI for the people

HITL is a mix and match approach that may help make ML both more efficient and approachable.

Paco Nathan is a unicorn. It’s a cliche, but gets the point across for someone who is equally versed in discussing AI with White House officials and Microsoft product managers, working on big data pipelines and organizing and part-taking in conferences such as Strata in his role as Director, Learning Group with O’Reilly Media.

Nathan has a mix of diverse background, hands-on involvement and broad vision that enables him to engage in all of those, having been active in AI, Data Science and Software Engineering for decades. The trigger for our discussion was his Human in the Loop (HITL) framework for machine learning (ML), presented in Strata EU.

HITL is a mix and match approach that may help make ML both more efficient and approchable. Nathan calls HITL a design pattern, and it combines technical approaches as well as management aspects.

HITL combines two common ML variants, supervised and unsupervised learning. In supervised learning, curated (labeled) datasets are used by ML experts to train algorithms by adjusting parameters, in order to make accurate predictions for incoming data. In unsupervised learning, the idea is that running lots of data through an algorithm will reveal some sort of structure.

The less common ML variant that HITL builds on is called semi-supervised, and an important special case of that is known as “active learning.” The idea is to take an ensemble of ML models, and let them “vote” on how to label each case of input data. When the models agree, their consensus gets used, typically as an automated approach.

When the models disagree or lack confidence, decision is delegated to human experts who handle the difficult edge cases. Choices made by experts are fed back to the system to iterate on training the ML models.

Nathan says active learning works well when you have have lots of inexpensive, unlabeled data — an abundance of data, where the cost of labeling itself is a major expense. This is a very common scenario for most organizations outside of the Big Tech circle, which is what makes it interesting.

But technology alone is not enough. What could be a realistic way to bring ML, AI, and automation to mid-market businesses?

AI for the people

In Nathan’s experience, most executives are struggling to grasp what the technology could do for them and identify suitable use cases. Especially for mid-market businesses, AI may seem like a far cry. But Nathan thinks they should start as soon as possible, and not look to outsource, for a number of reasons:

We are at a point where competition is heating up, and AI is key. Companies are happy to share code, but not data. The competition is going to be about data, who has the best data to use. If you’re still struggling to move data from one silo to another, it means you’re behind at least 2 or 3 years.

Better allocate resources now, because in 5 years there will already be the haves and have nots. The way most mid-market businesses get on board is by seeing, and sharing experiences with, early adopters in their industry. This gets them going, and they build confidence.

Getting your data management right is table stakes – you can’t talk about AI without this. Some people think they can just leapfrog to AI. I don’t think there will be a SaaS model for AI that does much beyond trivialize consumer use cases. “Alexa, book me a flight” is easy, but what about “Alexa, I want to learn about Kubernetes”? It will fall apart.

Not everything will be subscription based, and you have to consider that market leaders will not share the viable parts of their business. They may say so, but they don’t — there are many examples.

analytic-maturity.jpg

Gartner’s analytics maturity model may be a good starting point to explain and prepare the transition to AI. Image: Gartner

Nathan says Big Tech tends to automate use cases that are easy for them, which means they do not involve interaction with humans, because that’s not in their DNA. This is why he recommends that businesses work with second or third tier experts who are also aware of the business pain points.

And even if “outsourcing AI” was an option, it would still not be a great idea, for a number of reasons. Not just because of the bias in the datasets or the ethical reasons, but also because it is important to leverage knowledge within the organization:

“To get to artificial intelligence, you have to start with human intelligence. Knowledge does not come from IT, it comes from line of business and sales, and nobody outsources those.”

Leveraging uncertainty

Nathan says he has witnessed a similar situation in the early days of Data Science, and although he acknowledges there is some distance to cover, he believes executives will eventually get it:
Decision makers are used to making judgments. Any CEO understands statistics at a gut level, because that’s what they do every day. They may not know the math behind it, but the idea of collecting evidence, iterating on it and basing decisions on this is intuitive for executives.
Nathan says he believes in HITL because it’s based on two things.

First, it combines teams of machines and people. Currently, very few people in HR have even considered this, but Nathan thinks that going forward we’ll need things such as role descriptions for automation:

Many things we considered exclusively for people so far, will have to be considered for people and machines. If only for the sake of compliance, auditing, and continuity, these things have to be be taken very seriously.

In my experience all organizations have projects they do not embark upon because they do not have the resources. Once their processes are automated, people will be freed to embark on those.

expert.jpg

Leveraging human expertise alongside AI is a way to develop and use AI.

Which brings us to the second point about HITL. Nathan points out that we typically think of ML as a way to identify patterns and generalize using lots of data, but we may be about to witness a paradigm shift:

Rather than just recognize patterns, we can use ML to look at data and identify opportunities, by identifying uncertainty. This is not about risk — there’s no upside to risk, you just buy insurance. But if you can separate risk from uncertainty, you can profit, because that’s where the opportunities are.

With active learning, we have a way to identify where the uncertainties lie in our dataset. We can filter out the risk, and bring in the human experts to focus on the opportunity. We are starting to see companies that do just that, for example Stitch Fix.

I believe in this mixed model of augmenting experts. In any vertical, there is this proficiency threshold. You can achieve up to 80 percent proficiency, like someone who is good in that field. You can go up to say 95 percent, which is what experts achieve.

Beyond that, you get diminishing returns — chaos and churn, judgment calls and experts disagreeing with each other. These are areas of exploration, there are no perfectly right answers, but we can actually leverage that.

Content retrieved from: https://www.zdnet.com/article/human-in-the-loop-machine-learning-and-ai-for-the-people/.

Categories
knowledge connexions

Water data is the new oil: Using data to preserve water

Data and water do mix, apparently. Using data and analytics can lead to great benefits in water preservation.

We’ve all heard data is the new oil a thousand times by now. Arguably though, we can all live without data, or even oil, but there’s one thing we can’t do without: Water. Preserving water and catering to water quality is a necessity, and data can help do that.

On the occasion of World Water Day, ZDNet discussed the use of data to preserve water with Gary Wong. Wong is the Global Water Industry Principal for OSIsoft, and he was recently named one of the world’s 50 most impactful leaders in Water & Water Management.

Preserving water through data

Wong started his career with Metro Vancouver, in which his role was to manage Corporate Applications for the water utility. After eight years in this role, and using OSIsoft in that capacity, when he got an offer to join OSIsoft to take the lead in building the Global Water Industry branch, he did not turn it down.

But what is the Global Water Industry exactly? During the 20th century the world population tripled, while water use for human purposes multiplied sixfold. The most obvious uses of water for people are drinking, cooking, bathing, cleaning, and — for some — watering family food plots.

This domestic water use, though crucial, is only a small part of the total. Though usage varies worldwide, industries use at least twice as much water as households, mostly for cooling in the production of electricity. Far more water, something like 70 percent of total consumption, is needed for agriculture and natural environment purposes.

Still, Wong said, the Global Water Industry focuses on utilities and domestic use of water, at least as far as OSIsoft is concerned. The reason for this evident asymmetry has to do with business models. Simply put, utilities seem to be more willing to invest in analytics to improve their efficiency.

manilaslumswater.jpg

In Manila, not the entirety of the population has access to water, and there is huge amount of waste due to the old and inefficient infrastructure. Image: (Image: Cherry Wolf / Flickr)

The pressure to make better use of water will be mounting. We can hope to see this carrying over beyond utilities. But even in what is essentially a minority of overall water usage, OSIsoft claimed over 150 water and wastewater utilities rely on its PI System to manage operational data for over 250 million customers.

Success stories we discussed with Wong range from the White House Utility District in Tennesse, serving about 100.000 people, to Maynilad, the water company serving Manila, the most densely populated city in the world, with an estimated total of over 21 million people in its urban area.

White House Utility District pinpointed a leak spilling nearly 147 million gallons a year, reduced its costs by over $1 million and postponed $15 million to $20 million in upgrades. Maynilad was able to recover 640 million liters a day that otherwise would’ve been lost through leakage and reduced the cost of some repairs from $1.7 million to $1,250.

These are some pretty impressive feats, and Wong pointed out to all the benefits this could bring at scale in the metropoles of the world. The obvious question then is: How is this magic possible?

It’s not magic, it’s data integration

It’s simpler than you may think. Wong explained that for the most part, no new investment in infrastructure was needed. Utilities, like most industrial complexes, already have SCADA systems in place that enable them to collect vast amounts of data. But oftentimes it turns out they do not know, or care, to do anything particularly useful with that data.

“It’s not unusual that i will discuss with some utility and they will drop some numbers from their SCADA, like, we’re collecting a gadzillion data points per hour,” Wong said. “My answer to that is: Great, but what do you actually do with that?”

Unsurprisingly, it seems culture is a big part of this issue. Wong explained that utilities often operate in a non competitive environment. Wong thinks that lack of competitive pressure may not provide enough incentive for them to maximize their efficiency and develop a data-driven culture.

wateranalyticspipeline.png

Getting value out of analytics for water utilities is no different than usual. It’s a mix of infrastructure, integration, and culture. (Image: OSIsoft)

But there’s also the technical part, of course. As Wong explained, the main barrier utilities have to deal with is integration. Their SCADA systems may be old, not sufficiently documented, produce idiosyncratic data formats, and not necessarily designed to work with one another.

A layer that can aggregate and offer a unified view over that disparate data is a prerequisite for utilizing it — a big part of the value OSIsoft brings. OSIsoft’s PI comes with pre-built integrations to an array of SCADA systems, and it adds operational systems such as ERPs and other data sources to the mix.

This unglamorous, but sorely needed work, is what enables analytics that lead to benefits for the utilities. As OSIsoft has been focusing on the industrial sector since its inception, it has developed and instilled domain-specific knowledge in its products. The ability to handle time series data, or provide out of the box metrics for the domain, are good examples of this.

Advanced water management

Getting the basics in place can already bring huge benefits. But that’s not all there is to it. Wong also referred to more advanced scenarios.

For example, predictive models have been developed that enable utilities to forecast demand with an accuracy as high as 98 percent. These models utilize data collected via OSIsoft, combined with other data such as weather forecast, which are then fed to machine learning models in external applications.

Another thing that is possible based on data collected from disparate sources is automated control. Tracing leaks, or finding optimal infrastructure configuration and use, is one thing, and acting upon this is another. What if instead of generating alerts for a crew of engineers, these SCADA systems could be sent commands to self-adjust? That’s why they are there after all.

Wong said this is possible, but not something OSIsoft does. Machine learning falls under this category, as well. He did note, however, that OSIsoft works with an array of partners that offer this type of services to clients, and such scenarios have been implemented.

It seems OSIsoft has a somewhat inflexible business model itself, probably mirroring the industry it operates in to some extent. Wong added, however, that it definitely sees the value of infrastructure such as smart meters and IoT going forward, and it is already making use of it.

datainfluxwater.png

It takes a lot to make the most of what we usually take for granted: water. (Image: OSIsoft)

Some other areas in which data integration enables innovation is open data and citizen science. The flip side of utilities not operating in competitive environments is that they may enable innovation for others. Wong mentioned that datasets collected by utilities can be, and in some cases are, made available to third parties as open data.

This can trigger scenarios such as coordinating with other utilities, or enabling startups to offer insights into consumption patterns for consumers. More often than not, there is no clear mandate for utilities to make that data available. But, for Wong, this is part of the do-good mission of utilities, even though these assets require investment to acquire and maintain.

There are also cases in which utilities reach out to citizens to provide data. This can be as simple as a call to report anything out of the ordinary in terms of water color or taste through social media — or as advanced as utilizing citizen-made observation platforms for water quality.

One thing is certain, however: We need as much efficiency and clarity in water management as possible. Data and analytics can facilitate this to a great extent, and they can be used to achieve great benefits.

Content retrieved from: https://www.zdnet.com/article/water-data-is-the-new-oil-using-data-to-preserve-water/.

Categories
knowledge connexions

Artificial intelligence in the real world: What can it actually do?

How the cloud enables the AI revolution

AI is mainstream these days. The attention it gets and the feelings it provokes cover the whole gamut: from hands-on technical to business, from social science to pop culture, and from pragmatism to awe and bewilderment. Data and analytics are a prerequisite and an enabler for AI, and the boundaries between the two are getting increasingly blurred.

Many people and organizations from different backgrounds and with different goals are exploring these boundaries, and we’ve had the chance to converse with a couple of prominent figures in analytics and AI who share their insights.

“Deep stupidity”

Professor Mark Bishop is a lot of things: an academic with numerous publications on AI, the director of TCIDA (Tungsten Centre for Intelligent Data Analytics), and a thinker with his own view on why there are impenetrable barriers between deep minds and real minds.

Bishop recently presented on this topic in GOTO Berlin. His talk, intriguingly titled “Deep stupidity – what deep Neural Networks can and cannot do,” was featured in the Future of IT track and attracted widespread interest.

In short, Bishop argues that AI cannot become sentient, because computers don’t understand semantics, lack mathematical insight, and cannot experience phenomenal sensation — based on his own “Dancing with Pixies” reductium.

Bishop however is not some far-out academic with no connection to the real world. He does, when prompted, tend to refer to epistemology and ontology at a rate that far surpasses that of the average person. But he is also among the world’s leading deep learning experts, having being deeply involved in neural networks before it was cool.

“I was practically mocked when I announced this was going to be my thesis topic, and going from that to seeing it in mainstream news is quite the distance,” he notes.

His expertise has earned him more than recognition and a pet topic, however. It has also gotten him involved in a number of data-centric initiatives with some of the world’s leading enterprises. Bishop, about to wrap up his current engagement with Tungsten as TCIDA director, notes that going from academic research and up in the sky discussions to real-world problems is quite the distance as well.

“My team and myself were hired to work with Tungsten to add more intelligence in their SaaS offering. The idea was that our expertise would help get the most out of data collected from Tungsten’s invoicing solution. We would help them with transaction analysis, fraud detection, customer churn, and all sorts of advanced applications.

But we were dumbfounded to realize there was an array of real-world problems we had to address before embarking on such endeavors, like matching addresses. We never bothered with such things before — it’s mundane, somebody must have addressed the address issue already, right? Well, no. It’s actually a thorny issue that was not solved, so we had to address it.”

Injecting AI into the enterprise

download.png

Injecting AI into enterprise software is a promising way to move forward, but beware of the mundane before tackling the advanced

Steven Hillion, on the other hand, comes at this from a different angle. With a PhD in mathematics from Berkeley, he does not lack relevant academic background. But Hillion made the turn to industry a long time ago, driven by the desire to apply his knowledge to solve real-world problems. Having previously served as VP of analytics for Greenplum, Hillion co-founded Alpine Data, and now serves as its CPO.

Hillion believes that we’re currently in the “first generation” of enterprise AI: tools that, while absolutely helpful, are pretty mundane when it comes to the potential of AI. A few organizations have already moved to the second generation, which consists of a mix of tools and platforms that can operationalize data science — e.g. custom solutions like Morgan Stanley’s 3D Insights Platform or off the shelf solutions such as Salesforce’s Einstein.

In many fields, employees (or their bosses) determine the set of tasks to focus on each day. They log into an app, go through a checklist, generate a BI report, etc. In contrast, AI could use existing operational data to automatically serve up the highest priority (or most relevant, or most profitable) tasks that a specific employee needs to focus on that day, and deliver those tasks directly within the relevant application.

“Success will be found in making AI pervasive across apps and operations and in its ability to affect people’s work behavior to achieve larger business objectives. And, it’s a future which is closer than many people realize. This is exactly what we have been doing with a number of our clients, gradually injecting AI-powered features into the everyday workflow of users and making them more productive.

Of course, this isn’t easy. And in fact, the difficult aspect of getting value out of AI is as much in solving the more mundane issues, like security or data provisioning or address matching, as it is in working with complex algorithms.”

Know thy data — and algorithms

Artificial Intelligence Legal, ethical, and policy issues

Before handing over to AI overlords, it may help to actually understand how AI works

So, do androids dream of electric sheep, and does it matter for your organization? Although no definitive answers exist at this point, it is safe to say that both Bishop and Hillion seem to think this is not exactly the first thing we should be worried about. Data and algorithmic transparency on the other hand may be.

Case in point — Google’s presentation on deep learning preceding Bishop’s one on GOTO. The presentation, aptly titled “Tensorflow and deep learning, without a PhD,” did deliver what it promised. It was a step-by-step, hands-on tutorial on how to use Tensorflow, Google’s open source toolkit for deep learning, given by Robert Kubis, senior developer advocate for the Google Cloud Platform.

Expectedly, it was a full house. Unexpectedly, that changed dramatically as the talk progressed: by the end, the room was half empty, and a lukewarm applause greeted off Kubis. Bishop’s talk, by contrast, started with what seemed like a full house, and ended proving there could actually be more people packed in the room, with a roaring applause and an entourage for Bishop.

There is an array of possible explanations for this. Perhaps Bishop’s delivery style was more appealing than Kubis’ — videos of AI-generated art and Bladerunner references make for a lighter talk than a recipe-style “do A then B” tutorial.

Perhaps up in the sky discussions are more appealing than hands-on guides for yet another framework — even if that happens to be Google’s open source implementation of the technology that is supposed to change everything.

Or maybe the techies that attended GOTO just don’t get Tensorflow — with or without a PhD. In all likelihood, very few people in Kubis’ audience could really connect with the recipe-like instructions delivered and understand why they were supposed to take the steps described, or how the algorithm actually works.

And they are not the only ones. Romeo Kienzler, chief data scientist at IBM Watson IoT, admitted in a recent AI Meetup discussion: “We know deep learning works, and it works well, but we don’t exactly understand why or how.” The million dollar question is — does it matter?

After all, one could argue, not all developers need to know or care about the intrinsic details of QSort or Bubble Sort to use a sort function in their APIs — they just need to know how to call it and trust it works. Of course, they can always dig into commonly used sort algorithms, dissect them, replay and reconstruct them, thus building trust in the process.

Deep learning and machine learning on the other hand are a somewhat different beast. Their complexity and their way of digressing from conventional procedural algorithmic wisdom make them hard to approach. Coupled with vast amounts of data, this makes for opaque systems, and adding poor data quality to the mix only aggravates the issue.

It’s still early days for mainstream AI, but dealing with opaqueness may prove key to its adoption.

Content retrieved from: https://www.zdnet.com/article/artificial-intelligence-in-the-real-world-what-can-it-actually-do/.

Categories
knowledge connexions

Artificial intelligence in the real world: What can it actually do?

How the cloud enables the AI revolution

AI is mainstream these days. The attention it gets and the feelings it provokes cover the whole gamut: from hands-on technical to business, from social science to pop culture, and from pragmatism to awe and bewilderment. Data and analytics are a prerequisite and an enabler for AI, and the boundaries between the two are getting increasingly blurred.

Many people and organizations from different backgrounds and with different goals are exploring these boundaries, and we’ve had the chance to converse with a couple of prominent figures in analytics and AI who share their insights.

“Deep stupidity”

Professor Mark Bishop is a lot of things: an academic with numerous publications on AI, the director of TCIDA (Tungsten Centre for Intelligent Data Analytics), and a thinker with his own view on why there are impenetrable barriers between deep minds and real minds.

Bishop recently presented on this topic in GOTO Berlin. His talk, intriguingly titled “Deep stupidity – what deep Neural Networks can and cannot do,” was featured in the Future of IT track and attracted widespread interest.

In short, Bishop argues that AI cannot become sentient, because computers don’t understand semantics, lack mathematical insight, and cannot experience phenomenal sensation — based on his own “Dancing with Pixies” reductium.

Bishop however is not some far-out academic with no connection to the real world. He does, when prompted, tend to refer to epistemology and ontology at a rate that far surpasses that of the average person. But he is also among the world’s leading deep learning experts, having being deeply involved in neural networks before it was cool.

“I was practically mocked when I announced this was going to be my thesis topic, and going from that to seeing it in mainstream news is quite the distance,” he notes.

His expertise has earned him more than recognition and a pet topic, however. It has also gotten him involved in a number of data-centric initiatives with some of the world’s leading enterprises. Bishop, about to wrap up his current engagement with Tungsten as TCIDA director, notes that going from academic research and up in the sky discussions to real-world problems is quite the distance as well.

“My team and myself were hired to work with Tungsten to add more intelligence in their SaaS offering. The idea was that our expertise would help get the most out of data collected from Tungsten’s invoicing solution. We would help them with transaction analysis, fraud detection, customer churn, and all sorts of advanced applications.

But we were dumbfounded to realize there was an array of real-world problems we had to address before embarking on such endeavors, like matching addresses. We never bothered with such things before — it’s mundane, somebody must have addressed the address issue already, right? Well, no. It’s actually a thorny issue that was not solved, so we had to address it.”

Injecting AI into the enterprise

download.png

Injecting AI into enterprise software is a promising way to move forward, but beware of the mundane before tackling the advanced

Steven Hillion, on the other hand, comes at this from a different angle. With a PhD in mathematics from Berkeley, he does not lack relevant academic background. But Hillion made the turn to industry a long time ago, driven by the desire to apply his knowledge to solve real-world problems. Having previously served as VP of analytics for Greenplum, Hillion co-founded Alpine Data, and now serves as its CPO.

Hillion believes that we’re currently in the “first generation” of enterprise AI: tools that, while absolutely helpful, are pretty mundane when it comes to the potential of AI. A few organizations have already moved to the second generation, which consists of a mix of tools and platforms that can operationalize data science — e.g. custom solutions like Morgan Stanley’s 3D Insights Platform or off the shelf solutions such as Salesforce’s Einstein.

In many fields, employees (or their bosses) determine the set of tasks to focus on each day. They log into an app, go through a checklist, generate a BI report, etc. In contrast, AI could use existing operational data to automatically serve up the highest priority (or most relevant, or most profitable) tasks that a specific employee needs to focus on that day, and deliver those tasks directly within the relevant application.

“Success will be found in making AI pervasive across apps and operations and in its ability to affect people’s work behavior to achieve larger business objectives. And, it’s a future which is closer than many people realize. This is exactly what we have been doing with a number of our clients, gradually injecting AI-powered features into the everyday workflow of users and making them more productive.

Of course, this isn’t easy. And in fact, the difficult aspect of getting value out of AI is as much in solving the more mundane issues, like security or data provisioning or address matching, as it is in working with complex algorithms.”

Know thy data — and algorithms

Artificial Intelligence Legal, ethical, and policy issues

Before handing over to AI overlords, it may help to actually understand how AI works

So, do androids dream of electric sheep, and does it matter for your organization? Although no definitive answers exist at this point, it is safe to say that both Bishop and Hillion seem to think this is not exactly the first thing we should be worried about. Data and algorithmic transparency on the other hand may be.

Case in point — Google’s presentation on deep learning preceding Bishop’s one on GOTO. The presentation, aptly titled “Tensorflow and deep learning, without a PhD,” did deliver what it promised. It was a step-by-step, hands-on tutorial on how to use Tensorflow, Google’s open source toolkit for deep learning, given by Robert Kubis, senior developer advocate for the Google Cloud Platform.

Expectedly, it was a full house. Unexpectedly, that changed dramatically as the talk progressed: by the end, the room was half empty, and a lukewarm applause greeted off Kubis. Bishop’s talk, by contrast, started with what seemed like a full house, and ended proving there could actually be more people packed in the room, with a roaring applause and an entourage for Bishop.

There is an array of possible explanations for this. Perhaps Bishop’s delivery style was more appealing than Kubis’ — videos of AI-generated art and Bladerunner references make for a lighter talk than a recipe-style “do A then B” tutorial.

Perhaps up in the sky discussions are more appealing than hands-on guides for yet another framework — even if that happens to be Google’s open source implementation of the technology that is supposed to change everything.

Or maybe the techies that attended GOTO just don’t get Tensorflow — with or without a PhD. In all likelihood, very few people in Kubis’ audience could really connect with the recipe-like instructions delivered and understand why they were supposed to take the steps described, or how the algorithm actually works.

And they are not the only ones. Romeo Kienzler, chief data scientist at IBM Watson IoT, admitted in a recent AI Meetup discussion: “We know deep learning works, and it works well, but we don’t exactly understand why or how.” The million dollar question is — does it matter?

After all, one could argue, not all developers need to know or care about the intrinsic details of QSort or Bubble Sort to use a sort function in their APIs — they just need to know how to call it and trust it works. Of course, they can always dig into commonly used sort algorithms, dissect them, replay and reconstruct them, thus building trust in the process.

Deep learning and machine learning on the other hand are a somewhat different beast. Their complexity and their way of digressing from conventional procedural algorithmic wisdom make them hard to approach. Coupled with vast amounts of data, this makes for opaque systems, and adding poor data quality to the mix only aggravates the issue.

It’s still early days for mainstream AI, but dealing with opaqueness may prove key to its adoption.

Content retrieved from: https://www.zdnet.com/article/artificial-intelligence-in-the-real-world-what-can-it-actually-do/.

Categories
knowledge connexions

IBM’s Watson does healthcare: Data as the foundation for cognitive systems for population health

Watson is IBM’s big bet on AI, and healthcare is a prime domain for present and future applications. We take an inside look at Watson, why and how it can benefit healthcare, and what kind of data is used by whom in this process.

IBM’s big bet on Watson is all over the news. This week’s World of Watson event helped bring Watson to the limelight, with attendees from 110+ countries. If numbers impress you, Ginni Rometty’s number-dropping in WoW’s keynote should leave you impressed indeed.

Watson is meant to be positioned as the leader in a market worth $32 billion (cognitive systems), help organizations make better decisions worth an estimated $2 trillion, and make a difference in the lives of nearly one billion people through its 700 clients.

200 million of these people are consumers, and another 200 million are patients, but according to Teva, one of IBM’s major partners in healthcare, the “consumerization” of healthcare is the driving force behind its ongoing transformation: consumers expect to get everything here and now, in a way that is convenient, affordable, transparent and adjusted to their needs. They will not accept healthcare they do not understand, costs too much, and requires them to leave the comfort of their home too often.

Social determinants of health

Spyros Kotoulas, research manager for IBM Health and Person-Centric Knowledge Systems, says “we are therefore moving from treating a set of problems to treating a person. Traditionally, IT in healthcare is there to (a) record and share information and (b) provide tools to help users make better decisions, based on clinical evidence.

There is a critical, and increasingly visible, gap between what health outcomes are expected, based on the patient’s clinical evidence, and observed outcomes. This gap is known as the social determinants of health: a person’s socioeconomic status, family situation, social context, etc. These play a huge role in health outcomes.

8802-figure-1.png

Social determinants play a key role in people’s well-being, and big data enables us to keep track of them. Image: Schroeder, SA

The next logical step is to build systems that account for these social determinants, decision support systems that are based on a broader set of criteria and a broader set of tasks. For example, doing deep personalization of care plans, based on what has worked in the past for similar patients or guiding health professionals to seek the information that will make the biggest difference in their decisions.”

IBM’s core skillset is in computer science and AI, and Kotoulas manages a team of researchers and engineers with AI-related backgrounds (semantic web, deep learning, machine learning, ranking and recommendation, natural language processing, and healthcare decision support).

Social determinants, however, are a concept that needs social and medical science to be utilized, therefore a highly interdisciplinary approach is taken, working closely with domain experts and customers in order to validate the effectiveness of approaches.

For example, according to Kotoulas, “IBM is involved in the ProACT project, working closely with experts in psychology, nursing, and primary care, as well as customers, to develop and validate a new paradigm for integrated care. This paradigm integrates advanced analytics and IoT to advance self-management of multiple morbidities in the elderly population and spans the home, community, and secondary care environment as well as healthcare and social care.

Experts in healthcare play a much more important role than domain experts in traditional [business intelligence], as the domain is much more challenging, and maintaining a ‘human touch’ is critical. Through recent acquisitions (Explorys, Phytel, Truven, Merge), a significant number of health professionals have joined the ranks of IBM, in addition to world-class health innovators such as Paul Tang and Kyu Rhee.”

Using streaming IoT data for healthcare

For Kotoulas, “the key factors for health are clinical, socio-economic, and medical care related. For each of these, you can have data on multiple levels. Aggregate data such as neighborhood poverty levels (socioeconomic), or health services (quality of care) are not sensitive and relatively easy to get (e.g. from the census).

That data has been shown to play an important role in health outcomes, but to better understand a person, their own situation needs to be understood, as well as that of their social context (family members, informal carers, community members).”

socialdeterminantssources.png

Healthcare needs a wide array of data sources to function, and streaming IoT data is valuable for getting to-the-minute insights. Image: IBM

Healthcare experts like Teva can gather huge amounts of data from all kinds of sources, including biosensors. Biosensors help learn more about patients, and Watson enables them to understand their condition and make suggestions.

Kotoulas says, “IoT is a key ingredient to deliver on the promise of truly integrated care. What IoT brings to the table is the ability to continuously monitor patients in their own environment.

This has many advantages: readings at home may be different from readings in a formal care environment, the frequency in which we can get information is much higher, it gives a sense of empowerment and independence to patients and, in many cases, it is cheaper — what’s not to like?

Making sense of IoT data, and particularly multiple concurrent streams, is something that can come in waves. You can get some very useful data very easily, like mobility and step counts which are readily accessible from smartphones.

Some data requires additional inference, for example detecting social isolation from multiple streams. There are also streams that are much harder to interpret and need specialized hardware, combining multiple signals at real-time to monitor some conditions.”

Watson is accessible as a collection of services running on the IBM Cloud, and several of the services in Watson Developer Cloud originate from IBM Research. But Watson’s underpinnings are a topic in and by itself, so stay tuned for more.

Content retrieved from: https://www.zdnet.com/article/watson-does-healthcare-data-as-the-foundation-for-cognitive-systems-for-population-health/.

Categories
knowledge connexions

Habana, the AI chip innovator, promises top performance and efficiency

Habana is the best kept secret in AI chips. Designed from the ground up for machine learning workloads, it promises superior performance combined with power efficiency to revolutionize everything from data centers in the cloud to autonomous cars.

As data generation and accumulation accelerates, we’ve reached a tipping point where using machine learning just works. Using machine learning to train models that find patterns in data and make predictions based on those is applied to pretty much everything today. But data and models are just one part of the story.

Another part, equally important, is compute. Machine learning consists of two phases: Training and inference. In the training phase patterns are extracted, and machine learning models that capture them are created. In the inference phase, trained models are deployed and fed with new data in order to generate results.

Both of these phases require compute power. Not just any compute in fact, as it turns out CPUs are not really geared towards the specialized type of computation required for machine learning workloads. GPUs are currently the weapon of choice when it comes to machine learning workloads, but that may be about to change.

AI chips just got more interesting

GPU vendor Nvidia has reinvented itself as an AI chip company, coming up with new processors geared specifically towards machine learning workloads and dominating this market. But the boom in machine learning workloads has whetted the appetite of others players, as well.

Cloud vendors such as Google and AWS are working on their own AI chips. Intel is working on getting FPGA chips in shape to support machine learning. And upstarts are having a go at entering this market as well. GraphCore is the most high profile among them, with recent funding having catapulted it into unicorn territory, but it’s not the only one: Enter Habana.

Habana has been working on its own processor for AI since 2015. But as Eitan Medina, its CBO told us in a recent discussion, it has been doing so in stealth until recently: “Our motto is AI performance, not stories. We have been working under cover until September 2018”. David Dahan, Habana CEO, said that “among all AI semiconductor startups, Habana Labs is the first, and still the only one, which introduced a production-ready AI processor.”

As Medina explained, Habana was founded by CEO David Dahan and VP R&D Ran Halutz. Both Dahan and Halutz are semi-conductor industry veterans, and they have worked together for years in semiconductor companies CEVA and PrimeSense. The management team also includes CTO Shlomo Raikin, former Intel project architect.

Medina himself also has an engineering background: “Our team has a deep background in machine learning. If you Google topics such as quantization, you will find our names,” Medina said. And there’s no lack of funding or staff either.

Habana just closed a Round B financing round of $75 million, led by Intel Capital no less, which brings its total funding to $120 million. Habana has a headcount of 120 and is based in Tel Aviv, Israel, but also has offices and R&D in San Jose, US, Gdansk, Poland, and Beijing, China.

This looks solid. All these people, funds, and know-how have been set in motion by identifying the opportunity. Much like GraphCore, Habana’s Medina thinks that the AI chip race is far from over, and that GPUs may be dominating for the time being, but that’s about to change. Habana brings two key innovations to the table: Specialized processors for training and inference, and power efficiency.

Separating training and inference to deliver superior performance

Medina noted that starting with a clean sheet to design their processor, one of the key decisions made early on was to address training and inference separately. As these workloads have different needs, Medina said that treating them separately has enabled them to optimize performance for each setting: “For years, GPU vendors have offered new versions of GPUs. Now Nvidia seems to have realized they need to differentiate. We got this from the start.”

Habana offers two different processors: Goya, addressing inference; and Gaudi, addressing training. Medina said that Goya is used in production today, while Gaudi will be released in Q2 2019. We wondered what was the reason inference was addressed first. Was it because the architecture and requirements for inference are simpler?

Medina said that it was a strategic decision based on market signals. Medina noted that the lion’s share of inference workloads in the cloud still runs on CPUs. Therefore, he explained, Habana’s primary goal at this stage is to address these workloads as a drop-in replacement. Indeed, according to Medina, Habana’s clients at this point are to a large extent data center owners and cloud providers, as well as autonomous cars ventures.

The value proposition in both cases is primarily performance. According to benchmarks published by Habana, Goya is significantly faster than both Intel’s CPUs and Nvidia’s GPUs. Habana used the well-known RES-50 benchmark, and Medina explained the rationale was that RES-50 is the easiest to measure and compare, as it has less variables.

Medina said other architectures must make compromises:

“Even when asked to give up latency, throughput is below where we are. With GPUs / CPUs, if you want better performance, you need to group data input in big groups of batches to feed the processor. Then you need to wait till entire group is finished to get the results. These architectures need this, otherwise throughput will not be good. But big batches are not usable. We have super high efficiency even with small batch sizes.”

There are some notable points about these benchmarks. The first, Medina pointed out, is that their scale is logarithmic, which is needed to be able to accommodate Goya and the competition in the same charts. Hence the claim that “Habana smokes inference incumbents.” The second is that results become even more interesting if power efficiency is factored in.

Power efficiency and the software stack

Power efficiency is a metric used to measure how much power is needed per calculation in benchmarks. This is a very important parameter. It’s not enough to deliver superior performance alone, the cost of delivering this is just as important. A standard metric to measure processor performance is IPS, Instructions Per Second. But IPS/W, or IPS per Watt, is probably a better one, as it takes into account the cost of delivering performance.

Higher power efficiency is better in every possible way. Thinking about data centers and autonomous vehicles, minimizing the cost of electricity, and increasing autonomy are key requirements. And in the bigger picture, lowering carbon footprint, is a major concern for the planet. As Medina put it, “You should care about the environment, and you should care about your pocket.”

Goya’s value proposition for data centers is focused on this, also factoring in latency requirements. As Medina said, for a scenario of processing 45K images/second, three Goya cards can get results with a latency of 1,3 msec, replacing 169 CPU servers with a latency of 70 msec plus 16 Nvidia Tesla V100 with a latency of 2,5 msec with a total cost around $400,000. The message is clear: You can do more with less.

TPC, Habana’s Tensor Processor Core at the heart of Goya, supports different form factors, memory configurations, and PCIe cards, as well as mixed-precision numeric. It is also programmable in C, and accessible via what Habana calls the GEMM engine (General Matric Multiplication). This touches upon another key aspect of AI chips: The software stack, and integrations with existing machine learning frameworks.

As there is a slew of machine learning frameworks people use to build their models, supporting as many of them as seamlessly as possible is a key requirement. Goya supports models trained on any processor via an API called SynapseAI. At this point, SynapseAI supports TensorFlow, mxnet and ONNX, an emerging exchange format for deep learning models, and is working on adding support for PyTorch, and more.

Users should be able to deploy their models on Goya without having to fiddle with SynapseAI. For those who wish to tweak their models to include customizations, however, the option to do so is there, as well as IDE tools to support them. Medina said this low-level programming has been requested by clients who have developed custom ways of maximizing performance on their current setting and would like to replicate this on Goya.

The bigger picture

So, who are these clients, and how does one actually become a client? Medina said Habana has a sort of screening process for clients, as they are not yet at the point where they can ship massive quantities of Goya. Habana is sampling Goya to selected companies only at this time. That’s what’s written on the form you’ll have to fill in if you’re interested.

Not that Goya is a half-baked product, as it is used in production according to Medina. Specific names were not discussed, but yes, these include cloud vendors, so you can let your imagination run wild. Medina also emphasized its R&D on the hardware level for Goya is mostly done.

However, there is ongoing work to take things to the next level with 7 nanometer chips, plus work on  the Gaudi processor for training, which promises linear scalability. In addition, development of the software stack never ceases in order to optimize, add new features and support for more frameworks. Recently, Habana also published open source Linux drivers for Goya, which should help a lot considering Linux is what powers most data centers and embedded systems.

Habana, just like GraphCore, seems to have the potential to bring about a major disruption in the AI chip market and the world at large. Many of its premises are similar: A new architecture, experienced team, well funded, and looking to seize the opportunity. One obvious difference is on how they approach their public image, as GraphCore has been quite open about their work, while Habana was a relative unknown up to now.

And the obvious questions — which one is faster/better, which one will succeed, can they dethrone Nvidia — we simply don’t know. GraphCore has not published any benchmarks. Judging from an organization maturity point of view, Habana seems to be lagging at this point, but that does not necessarily mean much. One thing we can say is that this space is booming, and we can expect AI chip innovation to catalyze AI even further soon.

The takeaway from this, however, should be to make power efficiency a key aspect of the AI narrative going forward. Performance comes at a price, and this should be factored in.

Content retrieved from: https://www.zdnet.com/article/habana-the-ai-chip-innovator-promising-top-performance-and-efficiency/.

Categories
knowledge connexions

Habana, the AI chip innovator, promises top performance and efficiency

Habana is the best kept secret in AI chips. Designed from the ground up for machine learning workloads, it promises superior performance combined with power efficiency to revolutionize everything from data centers in the cloud to autonomous cars.

As data generation and accumulation accelerates, we’ve reached a tipping point where using machine learning just works. Using machine learning to train models that find patterns in data and make predictions based on those is applied to pretty much everything today. But data and models are just one part of the story.

Another part, equally important, is compute. Machine learning consists of two phases: Training and inference. In the training phase patterns are extracted, and machine learning models that capture them are created. In the inference phase, trained models are deployed and fed with new data in order to generate results.

Both of these phases require compute power. Not just any compute in fact, as it turns out CPUs are not really geared towards the specialized type of computation required for machine learning workloads. GPUs are currently the weapon of choice when it comes to machine learning workloads, but that may be about to change.

AI chips just got more interesting

GPU vendor Nvidia has reinvented itself as an AI chip company, coming up with new processors geared specifically towards machine learning workloads and dominating this market. But the boom in machine learning workloads has whetted the appetite of others players, as well.

Cloud vendors such as Google and AWS are working on their own AI chips. Intel is working on getting FPGA chips in shape to support machine learning. And upstarts are having a go at entering this market as well. GraphCore is the most high profile among them, with recent funding having catapulted it into unicorn territory, but it’s not the only one: Enter Habana.

Habana has been working on its own processor for AI since 2015. But as Eitan Medina, its CBO told us in a recent discussion, it has been doing so in stealth until recently: “Our motto is AI performance, not stories. We have been working under cover until September 2018”. David Dahan, Habana CEO, said that “among all AI semiconductor startups, Habana Labs is the first, and still the only one, which introduced a production-ready AI processor.”

As Medina explained, Habana was founded by CEO David Dahan and VP R&D Ran Halutz. Both Dahan and Halutz are semi-conductor industry veterans, and they have worked together for years in semiconductor companies CEVA and PrimeSense. The management team also includes CTO Shlomo Raikin, former Intel project architect.

Medina himself also has an engineering background: “Our team has a deep background in machine learning. If you Google topics such as quantization, you will find our names,” Medina said. And there’s no lack of funding or staff either.

Habana just closed a Round B financing round of $75 million, led by Intel Capital no less, which brings its total funding to $120 million. Habana has a headcount of 120 and is based in Tel Aviv, Israel, but also has offices and R&D in San Jose, US, Gdansk, Poland, and Beijing, China.

This looks solid. All these people, funds, and know-how have been set in motion by identifying the opportunity. Much like GraphCore, Habana’s Medina thinks that the AI chip race is far from over, and that GPUs may be dominating for the time being, but that’s about to change. Habana brings two key innovations to the table: Specialized processors for training and inference, and power efficiency.

Separating training and inference to deliver superior performance

Medina noted that starting with a clean sheet to design their processor, one of the key decisions made early on was to address training and inference separately. As these workloads have different needs, Medina said that treating them separately has enabled them to optimize performance for each setting: “For years, GPU vendors have offered new versions of GPUs. Now Nvidia seems to have realized they need to differentiate. We got this from the start.”

Habana offers two different processors: Goya, addressing inference; and Gaudi, addressing training. Medina said that Goya is used in production today, while Gaudi will be released in Q2 2019. We wondered what was the reason inference was addressed first. Was it because the architecture and requirements for inference are simpler?

Medina said that it was a strategic decision based on market signals. Medina noted that the lion’s share of inference workloads in the cloud still runs on CPUs. Therefore, he explained, Habana’s primary goal at this stage is to address these workloads as a drop-in replacement. Indeed, according to Medina, Habana’s clients at this point are to a large extent data center owners and cloud providers, as well as autonomous cars ventures.

The value proposition in both cases is primarily performance. According to benchmarks published by Habana, Goya is significantly faster than both Intel’s CPUs and Nvidia’s GPUs. Habana used the well-known RES-50 benchmark, and Medina explained the rationale was that RES-50 is the easiest to measure and compare, as it has less variables.

Medina said other architectures must make compromises:

“Even when asked to give up latency, throughput is below where we are. With GPUs / CPUs, if you want better performance, you need to group data input in big groups of batches to feed the processor. Then you need to wait till entire group is finished to get the results. These architectures need this, otherwise throughput will not be good. But big batches are not usable. We have super high efficiency even with small batch sizes.”

There are some notable points about these benchmarks. The first, Medina pointed out, is that their scale is logarithmic, which is needed to be able to accommodate Goya and the competition in the same charts. Hence the claim that “Habana smokes inference incumbents.” The second is that results become even more interesting if power efficiency is factored in.

Power efficiency and the software stack

Power efficiency is a metric used to measure how much power is needed per calculation in benchmarks. This is a very important parameter. It’s not enough to deliver superior performance alone, the cost of delivering this is just as important. A standard metric to measure processor performance is IPS, Instructions Per Second. But IPS/W, or IPS per Watt, is probably a better one, as it takes into account the cost of delivering performance.

Higher power efficiency is better in every possible way. Thinking about data centers and autonomous vehicles, minimizing the cost of electricity, and increasing autonomy are key requirements. And in the bigger picture, lowering carbon footprint, is a major concern for the planet. As Medina put it, “You should care about the environment, and you should care about your pocket.”

Goya’s value proposition for data centers is focused on this, also factoring in latency requirements. As Medina said, for a scenario of processing 45K images/second, three Goya cards can get results with a latency of 1,3 msec, replacing 169 CPU servers with a latency of 70 msec plus 16 Nvidia Tesla V100 with a latency of 2,5 msec with a total cost around $400,000. The message is clear: You can do more with less.

TPC, Habana’s Tensor Processor Core at the heart of Goya, supports different form factors, memory configurations, and PCIe cards, as well as mixed-precision numeric. It is also programmable in C, and accessible via what Habana calls the GEMM engine (General Matric Multiplication). This touches upon another key aspect of AI chips: The software stack, and integrations with existing machine learning frameworks.

As there is a slew of machine learning frameworks people use to build their models, supporting as many of them as seamlessly as possible is a key requirement. Goya supports models trained on any processor via an API called SynapseAI. At this point, SynapseAI supports TensorFlow, mxnet and ONNX, an emerging exchange format for deep learning models, and is working on adding support for PyTorch, and more.

Users should be able to deploy their models on Goya without having to fiddle with SynapseAI. For those who wish to tweak their models to include customizations, however, the option to do so is there, as well as IDE tools to support them. Medina said this low-level programming has been requested by clients who have developed custom ways of maximizing performance on their current setting and would like to replicate this on Goya.

The bigger picture

So, who are these clients, and how does one actually become a client? Medina said Habana has a sort of screening process for clients, as they are not yet at the point where they can ship massive quantities of Goya. Habana is sampling Goya to selected companies only at this time. That’s what’s written on the form you’ll have to fill in if you’re interested.

Not that Goya is a half-baked product, as it is used in production according to Medina. Specific names were not discussed, but yes, these include cloud vendors, so you can let your imagination run wild. Medina also emphasized its R&D on the hardware level for Goya is mostly done.

However, there is ongoing work to take things to the next level with 7 nanometer chips, plus work on  the Gaudi processor for training, which promises linear scalability. In addition, development of the software stack never ceases in order to optimize, add new features and support for more frameworks. Recently, Habana also published open source Linux drivers for Goya, which should help a lot considering Linux is what powers most data centers and embedded systems.

Habana, just like GraphCore, seems to have the potential to bring about a major disruption in the AI chip market and the world at large. Many of its premises are similar: A new architecture, experienced team, well funded, and looking to seize the opportunity. One obvious difference is on how they approach their public image, as GraphCore has been quite open about their work, while Habana was a relative unknown up to now.

And the obvious questions — which one is faster/better, which one will succeed, can they dethrone Nvidia — we simply don’t know. GraphCore has not published any benchmarks. Judging from an organization maturity point of view, Habana seems to be lagging at this point, but that does not necessarily mean much. One thing we can say is that this space is booming, and we can expect AI chip innovation to catalyze AI even further soon.

The takeaway from this, however, should be to make power efficiency a key aspect of the AI narrative going forward. Performance comes at a price, and this should be factored in.

Content retrieved from: https://www.zdnet.com/article/habana-the-ai-chip-innovator-promising-top-performance-and-efficiency/.