Categories
knowledge connexions

Human in the loop: Machine learning and AI for the people

HITL is a mix and match approach that may help make ML both more efficient and approachable.

Paco Nathan is a unicorn. It’s a cliche, but gets the point across for someone who is equally versed in discussing AI with White House officials and Microsoft product managers, working on big data pipelines and organizing and part-taking in conferences such as Strata in his role as Director, Learning Group with O’Reilly Media.

Nathan has a mix of diverse background, hands-on involvement and broad vision that enables him to engage in all of those, having been active in AI, Data Science and Software Engineering for decades. The trigger for our discussion was his Human in the Loop (HITL) framework for machine learning (ML), presented in Strata EU.

HITL is a mix and match approach that may help make ML both more efficient and approchable. Nathan calls HITL a design pattern, and it combines technical approaches as well as management aspects.

HITL combines two common ML variants, supervised and unsupervised learning. In supervised learning, curated (labeled) datasets are used by ML experts to train algorithms by adjusting parameters, in order to make accurate predictions for incoming data. In unsupervised learning, the idea is that running lots of data through an algorithm will reveal some sort of structure.

The less common ML variant that HITL builds on is called semi-supervised, and an important special case of that is known as “active learning.” The idea is to take an ensemble of ML models, and let them “vote” on how to label each case of input data. When the models agree, their consensus gets used, typically as an automated approach.

When the models disagree or lack confidence, decision is delegated to human experts who handle the difficult edge cases. Choices made by experts are fed back to the system to iterate on training the ML models.

Nathan says active learning works well when you have have lots of inexpensive, unlabeled data — an abundance of data, where the cost of labeling itself is a major expense. This is a very common scenario for most organizations outside of the Big Tech circle, which is what makes it interesting.

But technology alone is not enough. What could be a realistic way to bring ML, AI, and automation to mid-market businesses?

AI for the people

In Nathan’s experience, most executives are struggling to grasp what the technology could do for them and identify suitable use cases. Especially for mid-market businesses, AI may seem like a far cry. But Nathan thinks they should start as soon as possible, and not look to outsource, for a number of reasons:

We are at a point where competition is heating up, and AI is key. Companies are happy to share code, but not data. The competition is going to be about data, who has the best data to use. If you’re still struggling to move data from one silo to another, it means you’re behind at least 2 or 3 years.

Better allocate resources now, because in 5 years there will already be the haves and have nots. The way most mid-market businesses get on board is by seeing, and sharing experiences with, early adopters in their industry. This gets them going, and they build confidence.

Getting your data management right is table stakes – you can’t talk about AI without this. Some people think they can just leapfrog to AI. I don’t think there will be a SaaS model for AI that does much beyond trivialize consumer use cases. “Alexa, book me a flight” is easy, but what about “Alexa, I want to learn about Kubernetes”? It will fall apart.

Not everything will be subscription based, and you have to consider that market leaders will not share the viable parts of their business. They may say so, but they don’t — there are many examples.

analytic-maturity.jpg

Gartner’s analytics maturity model may be a good starting point to explain and prepare the transition to AI. Image: Gartner

Nathan says Big Tech tends to automate use cases that are easy for them, which means they do not involve interaction with humans, because that’s not in their DNA. This is why he recommends that businesses work with second or third tier experts who are also aware of the business pain points.

And even if “outsourcing AI” was an option, it would still not be a great idea, for a number of reasons. Not just because of the bias in the datasets or the ethical reasons, but also because it is important to leverage knowledge within the organization:

“To get to artificial intelligence, you have to start with human intelligence. Knowledge does not come from IT, it comes from line of business and sales, and nobody outsources those.”

Leveraging uncertainty

Nathan says he has witnessed a similar situation in the early days of Data Science, and although he acknowledges there is some distance to cover, he believes executives will eventually get it:
Decision makers are used to making judgments. Any CEO understands statistics at a gut level, because that’s what they do every day. They may not know the math behind it, but the idea of collecting evidence, iterating on it and basing decisions on this is intuitive for executives.
Nathan says he believes in HITL because it’s based on two things.

First, it combines teams of machines and people. Currently, very few people in HR have even considered this, but Nathan thinks that going forward we’ll need things such as role descriptions for automation:

Many things we considered exclusively for people so far, will have to be considered for people and machines. If only for the sake of compliance, auditing, and continuity, these things have to be be taken very seriously.

In my experience all organizations have projects they do not embark upon because they do not have the resources. Once their processes are automated, people will be freed to embark on those.

expert.jpg

Leveraging human expertise alongside AI is a way to develop and use AI.

Which brings us to the second point about HITL. Nathan points out that we typically think of ML as a way to identify patterns and generalize using lots of data, but we may be about to witness a paradigm shift:

Rather than just recognize patterns, we can use ML to look at data and identify opportunities, by identifying uncertainty. This is not about risk — there’s no upside to risk, you just buy insurance. But if you can separate risk from uncertainty, you can profit, because that’s where the opportunities are.

With active learning, we have a way to identify where the uncertainties lie in our dataset. We can filter out the risk, and bring in the human experts to focus on the opportunity. We are starting to see companies that do just that, for example Stitch Fix.

I believe in this mixed model of augmenting experts. In any vertical, there is this proficiency threshold. You can achieve up to 80 percent proficiency, like someone who is good in that field. You can go up to say 95 percent, which is what experts achieve.

Beyond that, you get diminishing returns — chaos and churn, judgment calls and experts disagreeing with each other. These are areas of exploration, there are no perfectly right answers, but we can actually leverage that.

Content retrieved from: https://www.zdnet.com/article/human-in-the-loop-machine-learning-and-ai-for-the-people/.

Categories
knowledge connexions

Human in the loop: Machine learning and AI for the people

HITL is a mix and match approach that may help make ML both more efficient and approachable.

Paco Nathan is a unicorn. It’s a cliche, but gets the point across for someone who is equally versed in discussing AI with White House officials and Microsoft product managers, working on big data pipelines and organizing and part-taking in conferences such as Strata in his role as Director, Learning Group with O’Reilly Media.

Nathan has a mix of diverse background, hands-on involvement and broad vision that enables him to engage in all of those, having been active in AI, Data Science and Software Engineering for decades. The trigger for our discussion was his Human in the Loop (HITL) framework for machine learning (ML), presented in Strata EU.

HITL is a mix and match approach that may help make ML both more efficient and approchable. Nathan calls HITL a design pattern, and it combines technical approaches as well as management aspects.

HITL combines two common ML variants, supervised and unsupervised learning. In supervised learning, curated (labeled) datasets are used by ML experts to train algorithms by adjusting parameters, in order to make accurate predictions for incoming data. In unsupervised learning, the idea is that running lots of data through an algorithm will reveal some sort of structure.

The less common ML variant that HITL builds on is called semi-supervised, and an important special case of that is known as “active learning.” The idea is to take an ensemble of ML models, and let them “vote” on how to label each case of input data. When the models agree, their consensus gets used, typically as an automated approach.

When the models disagree or lack confidence, decision is delegated to human experts who handle the difficult edge cases. Choices made by experts are fed back to the system to iterate on training the ML models.

Nathan says active learning works well when you have have lots of inexpensive, unlabeled data — an abundance of data, where the cost of labeling itself is a major expense. This is a very common scenario for most organizations outside of the Big Tech circle, which is what makes it interesting.

But technology alone is not enough. What could be a realistic way to bring ML, AI, and automation to mid-market businesses?

AI for the people

In Nathan’s experience, most executives are struggling to grasp what the technology could do for them and identify suitable use cases. Especially for mid-market businesses, AI may seem like a far cry. But Nathan thinks they should start as soon as possible, and not look to outsource, for a number of reasons:

We are at a point where competition is heating up, and AI is key. Companies are happy to share code, but not data. The competition is going to be about data, who has the best data to use. If you’re still struggling to move data from one silo to another, it means you’re behind at least 2 or 3 years.

Better allocate resources now, because in 5 years there will already be the haves and have nots. The way most mid-market businesses get on board is by seeing, and sharing experiences with, early adopters in their industry. This gets them going, and they build confidence.

Getting your data management right is table stakes – you can’t talk about AI without this. Some people think they can just leapfrog to AI. I don’t think there will be a SaaS model for AI that does much beyond trivialize consumer use cases. “Alexa, book me a flight” is easy, but what about “Alexa, I want to learn about Kubernetes”? It will fall apart.

Not everything will be subscription based, and you have to consider that market leaders will not share the viable parts of their business. They may say so, but they don’t — there are many examples.

analytic-maturity.jpg

Gartner’s analytics maturity model may be a good starting point to explain and prepare the transition to AI. Image: Gartner

Nathan says Big Tech tends to automate use cases that are easy for them, which means they do not involve interaction with humans, because that’s not in their DNA. This is why he recommends that businesses work with second or third tier experts who are also aware of the business pain points.

And even if “outsourcing AI” was an option, it would still not be a great idea, for a number of reasons. Not just because of the bias in the datasets or the ethical reasons, but also because it is important to leverage knowledge within the organization:

“To get to artificial intelligence, you have to start with human intelligence. Knowledge does not come from IT, it comes from line of business and sales, and nobody outsources those.”

Leveraging uncertainty

Nathan says he has witnessed a similar situation in the early days of Data Science, and although he acknowledges there is some distance to cover, he believes executives will eventually get it:
Decision makers are used to making judgments. Any CEO understands statistics at a gut level, because that’s what they do every day. They may not know the math behind it, but the idea of collecting evidence, iterating on it and basing decisions on this is intuitive for executives.
Nathan says he believes in HITL because it’s based on two things.

First, it combines teams of machines and people. Currently, very few people in HR have even considered this, but Nathan thinks that going forward we’ll need things such as role descriptions for automation:

Many things we considered exclusively for people so far, will have to be considered for people and machines. If only for the sake of compliance, auditing, and continuity, these things have to be be taken very seriously.

In my experience all organizations have projects they do not embark upon because they do not have the resources. Once their processes are automated, people will be freed to embark on those.

expert.jpg

Leveraging human expertise alongside AI is a way to develop and use AI.

Which brings us to the second point about HITL. Nathan points out that we typically think of ML as a way to identify patterns and generalize using lots of data, but we may be about to witness a paradigm shift:

Rather than just recognize patterns, we can use ML to look at data and identify opportunities, by identifying uncertainty. This is not about risk — there’s no upside to risk, you just buy insurance. But if you can separate risk from uncertainty, you can profit, because that’s where the opportunities are.

With active learning, we have a way to identify where the uncertainties lie in our dataset. We can filter out the risk, and bring in the human experts to focus on the opportunity. We are starting to see companies that do just that, for example Stitch Fix.

I believe in this mixed model of augmenting experts. In any vertical, there is this proficiency threshold. You can achieve up to 80 percent proficiency, like someone who is good in that field. You can go up to say 95 percent, which is what experts achieve.

Beyond that, you get diminishing returns — chaos and churn, judgment calls and experts disagreeing with each other. These are areas of exploration, there are no perfectly right answers, but we can actually leverage that.

Content retrieved from: https://www.zdnet.com/article/human-in-the-loop-machine-learning-and-ai-for-the-people/.

Categories
knowledge connexions

Water data is the new oil: Using data to preserve water

Data and water do mix, apparently. Using data and analytics can lead to great benefits in water preservation.

We’ve all heard data is the new oil a thousand times by now. Arguably though, we can all live without data, or even oil, but there’s one thing we can’t do without: Water. Preserving water and catering to water quality is a necessity, and data can help do that.

On the occasion of World Water Day, ZDNet discussed the use of data to preserve water with Gary Wong. Wong is the Global Water Industry Principal for OSIsoft, and he was recently named one of the world’s 50 most impactful leaders in Water & Water Management.

Preserving water through data

Wong started his career with Metro Vancouver, in which his role was to manage Corporate Applications for the water utility. After eight years in this role, and using OSIsoft in that capacity, when he got an offer to join OSIsoft to take the lead in building the Global Water Industry branch, he did not turn it down.

But what is the Global Water Industry exactly? During the 20th century the world population tripled, while water use for human purposes multiplied sixfold. The most obvious uses of water for people are drinking, cooking, bathing, cleaning, and — for some — watering family food plots.

This domestic water use, though crucial, is only a small part of the total. Though usage varies worldwide, industries use at least twice as much water as households, mostly for cooling in the production of electricity. Far more water, something like 70 percent of total consumption, is needed for agriculture and natural environment purposes.

Still, Wong said, the Global Water Industry focuses on utilities and domestic use of water, at least as far as OSIsoft is concerned. The reason for this evident asymmetry has to do with business models. Simply put, utilities seem to be more willing to invest in analytics to improve their efficiency.

manilaslumswater.jpg

In Manila, not the entirety of the population has access to water, and there is huge amount of waste due to the old and inefficient infrastructure. Image: (Image: Cherry Wolf / Flickr)

The pressure to make better use of water will be mounting. We can hope to see this carrying over beyond utilities. But even in what is essentially a minority of overall water usage, OSIsoft claimed over 150 water and wastewater utilities rely on its PI System to manage operational data for over 250 million customers.

Success stories we discussed with Wong range from the White House Utility District in Tennesse, serving about 100.000 people, to Maynilad, the water company serving Manila, the most densely populated city in the world, with an estimated total of over 21 million people in its urban area.

White House Utility District pinpointed a leak spilling nearly 147 million gallons a year, reduced its costs by over $1 million and postponed $15 million to $20 million in upgrades. Maynilad was able to recover 640 million liters a day that otherwise would’ve been lost through leakage and reduced the cost of some repairs from $1.7 million to $1,250.

These are some pretty impressive feats, and Wong pointed out to all the benefits this could bring at scale in the metropoles of the world. The obvious question then is: How is this magic possible?

It’s not magic, it’s data integration

It’s simpler than you may think. Wong explained that for the most part, no new investment in infrastructure was needed. Utilities, like most industrial complexes, already have SCADA systems in place that enable them to collect vast amounts of data. But oftentimes it turns out they do not know, or care, to do anything particularly useful with that data.

“It’s not unusual that i will discuss with some utility and they will drop some numbers from their SCADA, like, we’re collecting a gadzillion data points per hour,” Wong said. “My answer to that is: Great, but what do you actually do with that?”

Unsurprisingly, it seems culture is a big part of this issue. Wong explained that utilities often operate in a non competitive environment. Wong thinks that lack of competitive pressure may not provide enough incentive for them to maximize their efficiency and develop a data-driven culture.

wateranalyticspipeline.png

Getting value out of analytics for water utilities is no different than usual. It’s a mix of infrastructure, integration, and culture. (Image: OSIsoft)

But there’s also the technical part, of course. As Wong explained, the main barrier utilities have to deal with is integration. Their SCADA systems may be old, not sufficiently documented, produce idiosyncratic data formats, and not necessarily designed to work with one another.

A layer that can aggregate and offer a unified view over that disparate data is a prerequisite for utilizing it — a big part of the value OSIsoft brings. OSIsoft’s PI comes with pre-built integrations to an array of SCADA systems, and it adds operational systems such as ERPs and other data sources to the mix.

This unglamorous, but sorely needed work, is what enables analytics that lead to benefits for the utilities. As OSIsoft has been focusing on the industrial sector since its inception, it has developed and instilled domain-specific knowledge in its products. The ability to handle time series data, or provide out of the box metrics for the domain, are good examples of this.

Advanced water management

Getting the basics in place can already bring huge benefits. But that’s not all there is to it. Wong also referred to more advanced scenarios.

For example, predictive models have been developed that enable utilities to forecast demand with an accuracy as high as 98 percent. These models utilize data collected via OSIsoft, combined with other data such as weather forecast, which are then fed to machine learning models in external applications.

Another thing that is possible based on data collected from disparate sources is automated control. Tracing leaks, or finding optimal infrastructure configuration and use, is one thing, and acting upon this is another. What if instead of generating alerts for a crew of engineers, these SCADA systems could be sent commands to self-adjust? That’s why they are there after all.

Wong said this is possible, but not something OSIsoft does. Machine learning falls under this category, as well. He did note, however, that OSIsoft works with an array of partners that offer this type of services to clients, and such scenarios have been implemented.

It seems OSIsoft has a somewhat inflexible business model itself, probably mirroring the industry it operates in to some extent. Wong added, however, that it definitely sees the value of infrastructure such as smart meters and IoT going forward, and it is already making use of it.

datainfluxwater.png

It takes a lot to make the most of what we usually take for granted: water. (Image: OSIsoft)

Some other areas in which data integration enables innovation is open data and citizen science. The flip side of utilities not operating in competitive environments is that they may enable innovation for others. Wong mentioned that datasets collected by utilities can be, and in some cases are, made available to third parties as open data.

This can trigger scenarios such as coordinating with other utilities, or enabling startups to offer insights into consumption patterns for consumers. More often than not, there is no clear mandate for utilities to make that data available. But, for Wong, this is part of the do-good mission of utilities, even though these assets require investment to acquire and maintain.

There are also cases in which utilities reach out to citizens to provide data. This can be as simple as a call to report anything out of the ordinary in terms of water color or taste through social media — or as advanced as utilizing citizen-made observation platforms for water quality.

One thing is certain, however: We need as much efficiency and clarity in water management as possible. Data and analytics can facilitate this to a great extent, and they can be used to achieve great benefits.

Content retrieved from: https://www.zdnet.com/article/water-data-is-the-new-oil-using-data-to-preserve-water/.

Categories
knowledge connexions

Artificial intelligence in the real world: What can it actually do?

How the cloud enables the AI revolution

AI is mainstream these days. The attention it gets and the feelings it provokes cover the whole gamut: from hands-on technical to business, from social science to pop culture, and from pragmatism to awe and bewilderment. Data and analytics are a prerequisite and an enabler for AI, and the boundaries between the two are getting increasingly blurred.

Many people and organizations from different backgrounds and with different goals are exploring these boundaries, and we’ve had the chance to converse with a couple of prominent figures in analytics and AI who share their insights.

“Deep stupidity”

Professor Mark Bishop is a lot of things: an academic with numerous publications on AI, the director of TCIDA (Tungsten Centre for Intelligent Data Analytics), and a thinker with his own view on why there are impenetrable barriers between deep minds and real minds.

Bishop recently presented on this topic in GOTO Berlin. His talk, intriguingly titled “Deep stupidity – what deep Neural Networks can and cannot do,” was featured in the Future of IT track and attracted widespread interest.

In short, Bishop argues that AI cannot become sentient, because computers don’t understand semantics, lack mathematical insight, and cannot experience phenomenal sensation — based on his own “Dancing with Pixies” reductium.

Bishop however is not some far-out academic with no connection to the real world. He does, when prompted, tend to refer to epistemology and ontology at a rate that far surpasses that of the average person. But he is also among the world’s leading deep learning experts, having being deeply involved in neural networks before it was cool.

“I was practically mocked when I announced this was going to be my thesis topic, and going from that to seeing it in mainstream news is quite the distance,” he notes.

His expertise has earned him more than recognition and a pet topic, however. It has also gotten him involved in a number of data-centric initiatives with some of the world’s leading enterprises. Bishop, about to wrap up his current engagement with Tungsten as TCIDA director, notes that going from academic research and up in the sky discussions to real-world problems is quite the distance as well.

“My team and myself were hired to work with Tungsten to add more intelligence in their SaaS offering. The idea was that our expertise would help get the most out of data collected from Tungsten’s invoicing solution. We would help them with transaction analysis, fraud detection, customer churn, and all sorts of advanced applications.

But we were dumbfounded to realize there was an array of real-world problems we had to address before embarking on such endeavors, like matching addresses. We never bothered with such things before — it’s mundane, somebody must have addressed the address issue already, right? Well, no. It’s actually a thorny issue that was not solved, so we had to address it.”

Injecting AI into the enterprise

download.png

Injecting AI into enterprise software is a promising way to move forward, but beware of the mundane before tackling the advanced

Steven Hillion, on the other hand, comes at this from a different angle. With a PhD in mathematics from Berkeley, he does not lack relevant academic background. But Hillion made the turn to industry a long time ago, driven by the desire to apply his knowledge to solve real-world problems. Having previously served as VP of analytics for Greenplum, Hillion co-founded Alpine Data, and now serves as its CPO.

Hillion believes that we’re currently in the “first generation” of enterprise AI: tools that, while absolutely helpful, are pretty mundane when it comes to the potential of AI. A few organizations have already moved to the second generation, which consists of a mix of tools and platforms that can operationalize data science — e.g. custom solutions like Morgan Stanley’s 3D Insights Platform or off the shelf solutions such as Salesforce’s Einstein.

In many fields, employees (or their bosses) determine the set of tasks to focus on each day. They log into an app, go through a checklist, generate a BI report, etc. In contrast, AI could use existing operational data to automatically serve up the highest priority (or most relevant, or most profitable) tasks that a specific employee needs to focus on that day, and deliver those tasks directly within the relevant application.

“Success will be found in making AI pervasive across apps and operations and in its ability to affect people’s work behavior to achieve larger business objectives. And, it’s a future which is closer than many people realize. This is exactly what we have been doing with a number of our clients, gradually injecting AI-powered features into the everyday workflow of users and making them more productive.

Of course, this isn’t easy. And in fact, the difficult aspect of getting value out of AI is as much in solving the more mundane issues, like security or data provisioning or address matching, as it is in working with complex algorithms.”

Know thy data — and algorithms

Artificial Intelligence Legal, ethical, and policy issues

Before handing over to AI overlords, it may help to actually understand how AI works

So, do androids dream of electric sheep, and does it matter for your organization? Although no definitive answers exist at this point, it is safe to say that both Bishop and Hillion seem to think this is not exactly the first thing we should be worried about. Data and algorithmic transparency on the other hand may be.

Case in point — Google’s presentation on deep learning preceding Bishop’s one on GOTO. The presentation, aptly titled “Tensorflow and deep learning, without a PhD,” did deliver what it promised. It was a step-by-step, hands-on tutorial on how to use Tensorflow, Google’s open source toolkit for deep learning, given by Robert Kubis, senior developer advocate for the Google Cloud Platform.

Expectedly, it was a full house. Unexpectedly, that changed dramatically as the talk progressed: by the end, the room was half empty, and a lukewarm applause greeted off Kubis. Bishop’s talk, by contrast, started with what seemed like a full house, and ended proving there could actually be more people packed in the room, with a roaring applause and an entourage for Bishop.

There is an array of possible explanations for this. Perhaps Bishop’s delivery style was more appealing than Kubis’ — videos of AI-generated art and Bladerunner references make for a lighter talk than a recipe-style “do A then B” tutorial.

Perhaps up in the sky discussions are more appealing than hands-on guides for yet another framework — even if that happens to be Google’s open source implementation of the technology that is supposed to change everything.

Or maybe the techies that attended GOTO just don’t get Tensorflow — with or without a PhD. In all likelihood, very few people in Kubis’ audience could really connect with the recipe-like instructions delivered and understand why they were supposed to take the steps described, or how the algorithm actually works.

And they are not the only ones. Romeo Kienzler, chief data scientist at IBM Watson IoT, admitted in a recent AI Meetup discussion: “We know deep learning works, and it works well, but we don’t exactly understand why or how.” The million dollar question is — does it matter?

After all, one could argue, not all developers need to know or care about the intrinsic details of QSort or Bubble Sort to use a sort function in their APIs — they just need to know how to call it and trust it works. Of course, they can always dig into commonly used sort algorithms, dissect them, replay and reconstruct them, thus building trust in the process.

Deep learning and machine learning on the other hand are a somewhat different beast. Their complexity and their way of digressing from conventional procedural algorithmic wisdom make them hard to approach. Coupled with vast amounts of data, this makes for opaque systems, and adding poor data quality to the mix only aggravates the issue.

It’s still early days for mainstream AI, but dealing with opaqueness may prove key to its adoption.

Content retrieved from: https://www.zdnet.com/article/artificial-intelligence-in-the-real-world-what-can-it-actually-do/.

Categories
knowledge connexions

Artificial intelligence in the real world: What can it actually do?

How the cloud enables the AI revolution

AI is mainstream these days. The attention it gets and the feelings it provokes cover the whole gamut: from hands-on technical to business, from social science to pop culture, and from pragmatism to awe and bewilderment. Data and analytics are a prerequisite and an enabler for AI, and the boundaries between the two are getting increasingly blurred.

Many people and organizations from different backgrounds and with different goals are exploring these boundaries, and we’ve had the chance to converse with a couple of prominent figures in analytics and AI who share their insights.

“Deep stupidity”

Professor Mark Bishop is a lot of things: an academic with numerous publications on AI, the director of TCIDA (Tungsten Centre for Intelligent Data Analytics), and a thinker with his own view on why there are impenetrable barriers between deep minds and real minds.

Bishop recently presented on this topic in GOTO Berlin. His talk, intriguingly titled “Deep stupidity – what deep Neural Networks can and cannot do,” was featured in the Future of IT track and attracted widespread interest.

In short, Bishop argues that AI cannot become sentient, because computers don’t understand semantics, lack mathematical insight, and cannot experience phenomenal sensation — based on his own “Dancing with Pixies” reductium.

Bishop however is not some far-out academic with no connection to the real world. He does, when prompted, tend to refer to epistemology and ontology at a rate that far surpasses that of the average person. But he is also among the world’s leading deep learning experts, having being deeply involved in neural networks before it was cool.

“I was practically mocked when I announced this was going to be my thesis topic, and going from that to seeing it in mainstream news is quite the distance,” he notes.

His expertise has earned him more than recognition and a pet topic, however. It has also gotten him involved in a number of data-centric initiatives with some of the world’s leading enterprises. Bishop, about to wrap up his current engagement with Tungsten as TCIDA director, notes that going from academic research and up in the sky discussions to real-world problems is quite the distance as well.

“My team and myself were hired to work with Tungsten to add more intelligence in their SaaS offering. The idea was that our expertise would help get the most out of data collected from Tungsten’s invoicing solution. We would help them with transaction analysis, fraud detection, customer churn, and all sorts of advanced applications.

But we were dumbfounded to realize there was an array of real-world problems we had to address before embarking on such endeavors, like matching addresses. We never bothered with such things before — it’s mundane, somebody must have addressed the address issue already, right? Well, no. It’s actually a thorny issue that was not solved, so we had to address it.”

Injecting AI into the enterprise

download.png

Injecting AI into enterprise software is a promising way to move forward, but beware of the mundane before tackling the advanced

Steven Hillion, on the other hand, comes at this from a different angle. With a PhD in mathematics from Berkeley, he does not lack relevant academic background. But Hillion made the turn to industry a long time ago, driven by the desire to apply his knowledge to solve real-world problems. Having previously served as VP of analytics for Greenplum, Hillion co-founded Alpine Data, and now serves as its CPO.

Hillion believes that we’re currently in the “first generation” of enterprise AI: tools that, while absolutely helpful, are pretty mundane when it comes to the potential of AI. A few organizations have already moved to the second generation, which consists of a mix of tools and platforms that can operationalize data science — e.g. custom solutions like Morgan Stanley’s 3D Insights Platform or off the shelf solutions such as Salesforce’s Einstein.

In many fields, employees (or their bosses) determine the set of tasks to focus on each day. They log into an app, go through a checklist, generate a BI report, etc. In contrast, AI could use existing operational data to automatically serve up the highest priority (or most relevant, or most profitable) tasks that a specific employee needs to focus on that day, and deliver those tasks directly within the relevant application.

“Success will be found in making AI pervasive across apps and operations and in its ability to affect people’s work behavior to achieve larger business objectives. And, it’s a future which is closer than many people realize. This is exactly what we have been doing with a number of our clients, gradually injecting AI-powered features into the everyday workflow of users and making them more productive.

Of course, this isn’t easy. And in fact, the difficult aspect of getting value out of AI is as much in solving the more mundane issues, like security or data provisioning or address matching, as it is in working with complex algorithms.”

Know thy data — and algorithms

Artificial Intelligence Legal, ethical, and policy issues

Before handing over to AI overlords, it may help to actually understand how AI works

So, do androids dream of electric sheep, and does it matter for your organization? Although no definitive answers exist at this point, it is safe to say that both Bishop and Hillion seem to think this is not exactly the first thing we should be worried about. Data and algorithmic transparency on the other hand may be.

Case in point — Google’s presentation on deep learning preceding Bishop’s one on GOTO. The presentation, aptly titled “Tensorflow and deep learning, without a PhD,” did deliver what it promised. It was a step-by-step, hands-on tutorial on how to use Tensorflow, Google’s open source toolkit for deep learning, given by Robert Kubis, senior developer advocate for the Google Cloud Platform.

Expectedly, it was a full house. Unexpectedly, that changed dramatically as the talk progressed: by the end, the room was half empty, and a lukewarm applause greeted off Kubis. Bishop’s talk, by contrast, started with what seemed like a full house, and ended proving there could actually be more people packed in the room, with a roaring applause and an entourage for Bishop.

There is an array of possible explanations for this. Perhaps Bishop’s delivery style was more appealing than Kubis’ — videos of AI-generated art and Bladerunner references make for a lighter talk than a recipe-style “do A then B” tutorial.

Perhaps up in the sky discussions are more appealing than hands-on guides for yet another framework — even if that happens to be Google’s open source implementation of the technology that is supposed to change everything.

Or maybe the techies that attended GOTO just don’t get Tensorflow — with or without a PhD. In all likelihood, very few people in Kubis’ audience could really connect with the recipe-like instructions delivered and understand why they were supposed to take the steps described, or how the algorithm actually works.

And they are not the only ones. Romeo Kienzler, chief data scientist at IBM Watson IoT, admitted in a recent AI Meetup discussion: “We know deep learning works, and it works well, but we don’t exactly understand why or how.” The million dollar question is — does it matter?

After all, one could argue, not all developers need to know or care about the intrinsic details of QSort or Bubble Sort to use a sort function in their APIs — they just need to know how to call it and trust it works. Of course, they can always dig into commonly used sort algorithms, dissect them, replay and reconstruct them, thus building trust in the process.

Deep learning and machine learning on the other hand are a somewhat different beast. Their complexity and their way of digressing from conventional procedural algorithmic wisdom make them hard to approach. Coupled with vast amounts of data, this makes for opaque systems, and adding poor data quality to the mix only aggravates the issue.

It’s still early days for mainstream AI, but dealing with opaqueness may prove key to its adoption.

Content retrieved from: https://www.zdnet.com/article/artificial-intelligence-in-the-real-world-what-can-it-actually-do/.

Categories
knowledge connexions

IBM’s Watson does healthcare: Data as the foundation for cognitive systems for population health

Watson is IBM’s big bet on AI, and healthcare is a prime domain for present and future applications. We take an inside look at Watson, why and how it can benefit healthcare, and what kind of data is used by whom in this process.

IBM’s big bet on Watson is all over the news. This week’s World of Watson event helped bring Watson to the limelight, with attendees from 110+ countries. If numbers impress you, Ginni Rometty’s number-dropping in WoW’s keynote should leave you impressed indeed.

Watson is meant to be positioned as the leader in a market worth $32 billion (cognitive systems), help organizations make better decisions worth an estimated $2 trillion, and make a difference in the lives of nearly one billion people through its 700 clients.

200 million of these people are consumers, and another 200 million are patients, but according to Teva, one of IBM’s major partners in healthcare, the “consumerization” of healthcare is the driving force behind its ongoing transformation: consumers expect to get everything here and now, in a way that is convenient, affordable, transparent and adjusted to their needs. They will not accept healthcare they do not understand, costs too much, and requires them to leave the comfort of their home too often.

Social determinants of health

Spyros Kotoulas, research manager for IBM Health and Person-Centric Knowledge Systems, says “we are therefore moving from treating a set of problems to treating a person. Traditionally, IT in healthcare is there to (a) record and share information and (b) provide tools to help users make better decisions, based on clinical evidence.

There is a critical, and increasingly visible, gap between what health outcomes are expected, based on the patient’s clinical evidence, and observed outcomes. This gap is known as the social determinants of health: a person’s socioeconomic status, family situation, social context, etc. These play a huge role in health outcomes.

8802-figure-1.png

Social determinants play a key role in people’s well-being, and big data enables us to keep track of them. Image: Schroeder, SA

The next logical step is to build systems that account for these social determinants, decision support systems that are based on a broader set of criteria and a broader set of tasks. For example, doing deep personalization of care plans, based on what has worked in the past for similar patients or guiding health professionals to seek the information that will make the biggest difference in their decisions.”

IBM’s core skillset is in computer science and AI, and Kotoulas manages a team of researchers and engineers with AI-related backgrounds (semantic web, deep learning, machine learning, ranking and recommendation, natural language processing, and healthcare decision support).

Social determinants, however, are a concept that needs social and medical science to be utilized, therefore a highly interdisciplinary approach is taken, working closely with domain experts and customers in order to validate the effectiveness of approaches.

For example, according to Kotoulas, “IBM is involved in the ProACT project, working closely with experts in psychology, nursing, and primary care, as well as customers, to develop and validate a new paradigm for integrated care. This paradigm integrates advanced analytics and IoT to advance self-management of multiple morbidities in the elderly population and spans the home, community, and secondary care environment as well as healthcare and social care.

Experts in healthcare play a much more important role than domain experts in traditional [business intelligence], as the domain is much more challenging, and maintaining a ‘human touch’ is critical. Through recent acquisitions (Explorys, Phytel, Truven, Merge), a significant number of health professionals have joined the ranks of IBM, in addition to world-class health innovators such as Paul Tang and Kyu Rhee.”

Using streaming IoT data for healthcare

For Kotoulas, “the key factors for health are clinical, socio-economic, and medical care related. For each of these, you can have data on multiple levels. Aggregate data such as neighborhood poverty levels (socioeconomic), or health services (quality of care) are not sensitive and relatively easy to get (e.g. from the census).

That data has been shown to play an important role in health outcomes, but to better understand a person, their own situation needs to be understood, as well as that of their social context (family members, informal carers, community members).”

socialdeterminantssources.png

Healthcare needs a wide array of data sources to function, and streaming IoT data is valuable for getting to-the-minute insights. Image: IBM

Healthcare experts like Teva can gather huge amounts of data from all kinds of sources, including biosensors. Biosensors help learn more about patients, and Watson enables them to understand their condition and make suggestions.

Kotoulas says, “IoT is a key ingredient to deliver on the promise of truly integrated care. What IoT brings to the table is the ability to continuously monitor patients in their own environment.

This has many advantages: readings at home may be different from readings in a formal care environment, the frequency in which we can get information is much higher, it gives a sense of empowerment and independence to patients and, in many cases, it is cheaper — what’s not to like?

Making sense of IoT data, and particularly multiple concurrent streams, is something that can come in waves. You can get some very useful data very easily, like mobility and step counts which are readily accessible from smartphones.

Some data requires additional inference, for example detecting social isolation from multiple streams. There are also streams that are much harder to interpret and need specialized hardware, combining multiple signals at real-time to monitor some conditions.”

Watson is accessible as a collection of services running on the IBM Cloud, and several of the services in Watson Developer Cloud originate from IBM Research. But Watson’s underpinnings are a topic in and by itself, so stay tuned for more.

Content retrieved from: https://www.zdnet.com/article/watson-does-healthcare-data-as-the-foundation-for-cognitive-systems-for-population-health/.

Categories
knowledge connexions

Habana, the AI chip innovator, promises top performance and efficiency

Habana is the best kept secret in AI chips. Designed from the ground up for machine learning workloads, it promises superior performance combined with power efficiency to revolutionize everything from data centers in the cloud to autonomous cars.

As data generation and accumulation accelerates, we’ve reached a tipping point where using machine learning just works. Using machine learning to train models that find patterns in data and make predictions based on those is applied to pretty much everything today. But data and models are just one part of the story.

Another part, equally important, is compute. Machine learning consists of two phases: Training and inference. In the training phase patterns are extracted, and machine learning models that capture them are created. In the inference phase, trained models are deployed and fed with new data in order to generate results.

Both of these phases require compute power. Not just any compute in fact, as it turns out CPUs are not really geared towards the specialized type of computation required for machine learning workloads. GPUs are currently the weapon of choice when it comes to machine learning workloads, but that may be about to change.

AI chips just got more interesting

GPU vendor Nvidia has reinvented itself as an AI chip company, coming up with new processors geared specifically towards machine learning workloads and dominating this market. But the boom in machine learning workloads has whetted the appetite of others players, as well.

Cloud vendors such as Google and AWS are working on their own AI chips. Intel is working on getting FPGA chips in shape to support machine learning. And upstarts are having a go at entering this market as well. GraphCore is the most high profile among them, with recent funding having catapulted it into unicorn territory, but it’s not the only one: Enter Habana.

Habana has been working on its own processor for AI since 2015. But as Eitan Medina, its CBO told us in a recent discussion, it has been doing so in stealth until recently: “Our motto is AI performance, not stories. We have been working under cover until September 2018”. David Dahan, Habana CEO, said that “among all AI semiconductor startups, Habana Labs is the first, and still the only one, which introduced a production-ready AI processor.”

As Medina explained, Habana was founded by CEO David Dahan and VP R&D Ran Halutz. Both Dahan and Halutz are semi-conductor industry veterans, and they have worked together for years in semiconductor companies CEVA and PrimeSense. The management team also includes CTO Shlomo Raikin, former Intel project architect.

Medina himself also has an engineering background: “Our team has a deep background in machine learning. If you Google topics such as quantization, you will find our names,” Medina said. And there’s no lack of funding or staff either.

Habana just closed a Round B financing round of $75 million, led by Intel Capital no less, which brings its total funding to $120 million. Habana has a headcount of 120 and is based in Tel Aviv, Israel, but also has offices and R&D in San Jose, US, Gdansk, Poland, and Beijing, China.

This looks solid. All these people, funds, and know-how have been set in motion by identifying the opportunity. Much like GraphCore, Habana’s Medina thinks that the AI chip race is far from over, and that GPUs may be dominating for the time being, but that’s about to change. Habana brings two key innovations to the table: Specialized processors for training and inference, and power efficiency.

Separating training and inference to deliver superior performance

Medina noted that starting with a clean sheet to design their processor, one of the key decisions made early on was to address training and inference separately. As these workloads have different needs, Medina said that treating them separately has enabled them to optimize performance for each setting: “For years, GPU vendors have offered new versions of GPUs. Now Nvidia seems to have realized they need to differentiate. We got this from the start.”

Habana offers two different processors: Goya, addressing inference; and Gaudi, addressing training. Medina said that Goya is used in production today, while Gaudi will be released in Q2 2019. We wondered what was the reason inference was addressed first. Was it because the architecture and requirements for inference are simpler?

Medina said that it was a strategic decision based on market signals. Medina noted that the lion’s share of inference workloads in the cloud still runs on CPUs. Therefore, he explained, Habana’s primary goal at this stage is to address these workloads as a drop-in replacement. Indeed, according to Medina, Habana’s clients at this point are to a large extent data center owners and cloud providers, as well as autonomous cars ventures.

The value proposition in both cases is primarily performance. According to benchmarks published by Habana, Goya is significantly faster than both Intel’s CPUs and Nvidia’s GPUs. Habana used the well-known RES-50 benchmark, and Medina explained the rationale was that RES-50 is the easiest to measure and compare, as it has less variables.

Medina said other architectures must make compromises:

“Even when asked to give up latency, throughput is below where we are. With GPUs / CPUs, if you want better performance, you need to group data input in big groups of batches to feed the processor. Then you need to wait till entire group is finished to get the results. These architectures need this, otherwise throughput will not be good. But big batches are not usable. We have super high efficiency even with small batch sizes.”

There are some notable points about these benchmarks. The first, Medina pointed out, is that their scale is logarithmic, which is needed to be able to accommodate Goya and the competition in the same charts. Hence the claim that “Habana smokes inference incumbents.” The second is that results become even more interesting if power efficiency is factored in.

Power efficiency and the software stack

Power efficiency is a metric used to measure how much power is needed per calculation in benchmarks. This is a very important parameter. It’s not enough to deliver superior performance alone, the cost of delivering this is just as important. A standard metric to measure processor performance is IPS, Instructions Per Second. But IPS/W, or IPS per Watt, is probably a better one, as it takes into account the cost of delivering performance.

Higher power efficiency is better in every possible way. Thinking about data centers and autonomous vehicles, minimizing the cost of electricity, and increasing autonomy are key requirements. And in the bigger picture, lowering carbon footprint, is a major concern for the planet. As Medina put it, “You should care about the environment, and you should care about your pocket.”

Goya’s value proposition for data centers is focused on this, also factoring in latency requirements. As Medina said, for a scenario of processing 45K images/second, three Goya cards can get results with a latency of 1,3 msec, replacing 169 CPU servers with a latency of 70 msec plus 16 Nvidia Tesla V100 with a latency of 2,5 msec with a total cost around $400,000. The message is clear: You can do more with less.

TPC, Habana’s Tensor Processor Core at the heart of Goya, supports different form factors, memory configurations, and PCIe cards, as well as mixed-precision numeric. It is also programmable in C, and accessible via what Habana calls the GEMM engine (General Matric Multiplication). This touches upon another key aspect of AI chips: The software stack, and integrations with existing machine learning frameworks.

As there is a slew of machine learning frameworks people use to build their models, supporting as many of them as seamlessly as possible is a key requirement. Goya supports models trained on any processor via an API called SynapseAI. At this point, SynapseAI supports TensorFlow, mxnet and ONNX, an emerging exchange format for deep learning models, and is working on adding support for PyTorch, and more.

Users should be able to deploy their models on Goya without having to fiddle with SynapseAI. For those who wish to tweak their models to include customizations, however, the option to do so is there, as well as IDE tools to support them. Medina said this low-level programming has been requested by clients who have developed custom ways of maximizing performance on their current setting and would like to replicate this on Goya.

The bigger picture

So, who are these clients, and how does one actually become a client? Medina said Habana has a sort of screening process for clients, as they are not yet at the point where they can ship massive quantities of Goya. Habana is sampling Goya to selected companies only at this time. That’s what’s written on the form you’ll have to fill in if you’re interested.

Not that Goya is a half-baked product, as it is used in production according to Medina. Specific names were not discussed, but yes, these include cloud vendors, so you can let your imagination run wild. Medina also emphasized its R&D on the hardware level for Goya is mostly done.

However, there is ongoing work to take things to the next level with 7 nanometer chips, plus work on  the Gaudi processor for training, which promises linear scalability. In addition, development of the software stack never ceases in order to optimize, add new features and support for more frameworks. Recently, Habana also published open source Linux drivers for Goya, which should help a lot considering Linux is what powers most data centers and embedded systems.

Habana, just like GraphCore, seems to have the potential to bring about a major disruption in the AI chip market and the world at large. Many of its premises are similar: A new architecture, experienced team, well funded, and looking to seize the opportunity. One obvious difference is on how they approach their public image, as GraphCore has been quite open about their work, while Habana was a relative unknown up to now.

And the obvious questions — which one is faster/better, which one will succeed, can they dethrone Nvidia — we simply don’t know. GraphCore has not published any benchmarks. Judging from an organization maturity point of view, Habana seems to be lagging at this point, but that does not necessarily mean much. One thing we can say is that this space is booming, and we can expect AI chip innovation to catalyze AI even further soon.

The takeaway from this, however, should be to make power efficiency a key aspect of the AI narrative going forward. Performance comes at a price, and this should be factored in.

Content retrieved from: https://www.zdnet.com/article/habana-the-ai-chip-innovator-promising-top-performance-and-efficiency/.

Categories
knowledge connexions

Habana, the AI chip innovator, promises top performance and efficiency

Habana is the best kept secret in AI chips. Designed from the ground up for machine learning workloads, it promises superior performance combined with power efficiency to revolutionize everything from data centers in the cloud to autonomous cars.

As data generation and accumulation accelerates, we’ve reached a tipping point where using machine learning just works. Using machine learning to train models that find patterns in data and make predictions based on those is applied to pretty much everything today. But data and models are just one part of the story.

Another part, equally important, is compute. Machine learning consists of two phases: Training and inference. In the training phase patterns are extracted, and machine learning models that capture them are created. In the inference phase, trained models are deployed and fed with new data in order to generate results.

Both of these phases require compute power. Not just any compute in fact, as it turns out CPUs are not really geared towards the specialized type of computation required for machine learning workloads. GPUs are currently the weapon of choice when it comes to machine learning workloads, but that may be about to change.

AI chips just got more interesting

GPU vendor Nvidia has reinvented itself as an AI chip company, coming up with new processors geared specifically towards machine learning workloads and dominating this market. But the boom in machine learning workloads has whetted the appetite of others players, as well.

Cloud vendors such as Google and AWS are working on their own AI chips. Intel is working on getting FPGA chips in shape to support machine learning. And upstarts are having a go at entering this market as well. GraphCore is the most high profile among them, with recent funding having catapulted it into unicorn territory, but it’s not the only one: Enter Habana.

Habana has been working on its own processor for AI since 2015. But as Eitan Medina, its CBO told us in a recent discussion, it has been doing so in stealth until recently: “Our motto is AI performance, not stories. We have been working under cover until September 2018”. David Dahan, Habana CEO, said that “among all AI semiconductor startups, Habana Labs is the first, and still the only one, which introduced a production-ready AI processor.”

As Medina explained, Habana was founded by CEO David Dahan and VP R&D Ran Halutz. Both Dahan and Halutz are semi-conductor industry veterans, and they have worked together for years in semiconductor companies CEVA and PrimeSense. The management team also includes CTO Shlomo Raikin, former Intel project architect.

Medina himself also has an engineering background: “Our team has a deep background in machine learning. If you Google topics such as quantization, you will find our names,” Medina said. And there’s no lack of funding or staff either.

Habana just closed a Round B financing round of $75 million, led by Intel Capital no less, which brings its total funding to $120 million. Habana has a headcount of 120 and is based in Tel Aviv, Israel, but also has offices and R&D in San Jose, US, Gdansk, Poland, and Beijing, China.

This looks solid. All these people, funds, and know-how have been set in motion by identifying the opportunity. Much like GraphCore, Habana’s Medina thinks that the AI chip race is far from over, and that GPUs may be dominating for the time being, but that’s about to change. Habana brings two key innovations to the table: Specialized processors for training and inference, and power efficiency.

Separating training and inference to deliver superior performance

Medina noted that starting with a clean sheet to design their processor, one of the key decisions made early on was to address training and inference separately. As these workloads have different needs, Medina said that treating them separately has enabled them to optimize performance for each setting: “For years, GPU vendors have offered new versions of GPUs. Now Nvidia seems to have realized they need to differentiate. We got this from the start.”

Habana offers two different processors: Goya, addressing inference; and Gaudi, addressing training. Medina said that Goya is used in production today, while Gaudi will be released in Q2 2019. We wondered what was the reason inference was addressed first. Was it because the architecture and requirements for inference are simpler?

Medina said that it was a strategic decision based on market signals. Medina noted that the lion’s share of inference workloads in the cloud still runs on CPUs. Therefore, he explained, Habana’s primary goal at this stage is to address these workloads as a drop-in replacement. Indeed, according to Medina, Habana’s clients at this point are to a large extent data center owners and cloud providers, as well as autonomous cars ventures.

The value proposition in both cases is primarily performance. According to benchmarks published by Habana, Goya is significantly faster than both Intel’s CPUs and Nvidia’s GPUs. Habana used the well-known RES-50 benchmark, and Medina explained the rationale was that RES-50 is the easiest to measure and compare, as it has less variables.

Medina said other architectures must make compromises:

“Even when asked to give up latency, throughput is below where we are. With GPUs / CPUs, if you want better performance, you need to group data input in big groups of batches to feed the processor. Then you need to wait till entire group is finished to get the results. These architectures need this, otherwise throughput will not be good. But big batches are not usable. We have super high efficiency even with small batch sizes.”

There are some notable points about these benchmarks. The first, Medina pointed out, is that their scale is logarithmic, which is needed to be able to accommodate Goya and the competition in the same charts. Hence the claim that “Habana smokes inference incumbents.” The second is that results become even more interesting if power efficiency is factored in.

Power efficiency and the software stack

Power efficiency is a metric used to measure how much power is needed per calculation in benchmarks. This is a very important parameter. It’s not enough to deliver superior performance alone, the cost of delivering this is just as important. A standard metric to measure processor performance is IPS, Instructions Per Second. But IPS/W, or IPS per Watt, is probably a better one, as it takes into account the cost of delivering performance.

Higher power efficiency is better in every possible way. Thinking about data centers and autonomous vehicles, minimizing the cost of electricity, and increasing autonomy are key requirements. And in the bigger picture, lowering carbon footprint, is a major concern for the planet. As Medina put it, “You should care about the environment, and you should care about your pocket.”

Goya’s value proposition for data centers is focused on this, also factoring in latency requirements. As Medina said, for a scenario of processing 45K images/second, three Goya cards can get results with a latency of 1,3 msec, replacing 169 CPU servers with a latency of 70 msec plus 16 Nvidia Tesla V100 with a latency of 2,5 msec with a total cost around $400,000. The message is clear: You can do more with less.

TPC, Habana’s Tensor Processor Core at the heart of Goya, supports different form factors, memory configurations, and PCIe cards, as well as mixed-precision numeric. It is also programmable in C, and accessible via what Habana calls the GEMM engine (General Matric Multiplication). This touches upon another key aspect of AI chips: The software stack, and integrations with existing machine learning frameworks.

As there is a slew of machine learning frameworks people use to build their models, supporting as many of them as seamlessly as possible is a key requirement. Goya supports models trained on any processor via an API called SynapseAI. At this point, SynapseAI supports TensorFlow, mxnet and ONNX, an emerging exchange format for deep learning models, and is working on adding support for PyTorch, and more.

Users should be able to deploy their models on Goya without having to fiddle with SynapseAI. For those who wish to tweak their models to include customizations, however, the option to do so is there, as well as IDE tools to support them. Medina said this low-level programming has been requested by clients who have developed custom ways of maximizing performance on their current setting and would like to replicate this on Goya.

The bigger picture

So, who are these clients, and how does one actually become a client? Medina said Habana has a sort of screening process for clients, as they are not yet at the point where they can ship massive quantities of Goya. Habana is sampling Goya to selected companies only at this time. That’s what’s written on the form you’ll have to fill in if you’re interested.

Not that Goya is a half-baked product, as it is used in production according to Medina. Specific names were not discussed, but yes, these include cloud vendors, so you can let your imagination run wild. Medina also emphasized its R&D on the hardware level for Goya is mostly done.

However, there is ongoing work to take things to the next level with 7 nanometer chips, plus work on  the Gaudi processor for training, which promises linear scalability. In addition, development of the software stack never ceases in order to optimize, add new features and support for more frameworks. Recently, Habana also published open source Linux drivers for Goya, which should help a lot considering Linux is what powers most data centers and embedded systems.

Habana, just like GraphCore, seems to have the potential to bring about a major disruption in the AI chip market and the world at large. Many of its premises are similar: A new architecture, experienced team, well funded, and looking to seize the opportunity. One obvious difference is on how they approach their public image, as GraphCore has been quite open about their work, while Habana was a relative unknown up to now.

And the obvious questions — which one is faster/better, which one will succeed, can they dethrone Nvidia — we simply don’t know. GraphCore has not published any benchmarks. Judging from an organization maturity point of view, Habana seems to be lagging at this point, but that does not necessarily mean much. One thing we can say is that this space is booming, and we can expect AI chip innovation to catalyze AI even further soon.

The takeaway from this, however, should be to make power efficiency a key aspect of the AI narrative going forward. Performance comes at a price, and this should be factored in.

Content retrieved from: https://www.zdnet.com/article/habana-the-ai-chip-innovator-promising-top-performance-and-efficiency/.

Categories
knowledge connexions

Salesforce Research: Knowledge graphs and machine learning to power Einstein

Explainable AI in real life could mean Einstein not just answering your questions, but also providing justification. Advancing the state of the art in natural language processing is done on the intersection of graphs and machine learning.

A super geeky topic, which could have super important repercussions in the real world. That description could very well fit anything from cold fusion to knowledge graphs, so a bit of unpacking is in order. (Hint: it’s about Salesforce, and Salesforce is not into cold fusion as far as we know.)

If you’re into science, chances are you know arXiv.org. arXiv is a repository of electronic publication preprints for scientific papers. In other words, it’s where cutting edge research often appears first. Some months back, a publication from researchers from Salesforce appeared in arXiv, titled “Multi-Hop Knowledge Graph Reasoning with Reward Shaping.”

The paper elaborates on a technique for using knowledge graphs with machine learning; specifically, a branch of machine learning called reinforcement learning. This is something that holds great promise as a way to get the best of both worlds: Curated, top-down knowledge representation (knowledge graphs), and emergent, bottom-up pattern recognition (machine learning).

This seemingly dry topic piqued our interest for a number of reasons, not the least of which was the prospect of seeing this being applied by Salesforce. Xi Victoria Lin, research scientist at Salesforce and the paper’s primary author, was kind enough to answer our questions.

Salesforce Research: it’s all about answering questions

To start with the obvious, the fact that this paper was published says a lot in and by itself. Salesforce presumably faces the same issue everyone else is facing in staffing their research these days: the boom in the applicability of machine learning in real-world problems means there is a ongoing race to attract and retain researchers.

  
People in the research community have an ethos of sharing their accomplishments with the world by publishing in conferences and journals. That, presumably, has a lot to do with why we are seeing a number of those publications lately coming from places such as Salesforce.

The paper, presented by Lin in the 2018 Conference on Empirical Methods in Natural Language Processing (NLP), was well received. The authors have also released the source code on Github. But what is that all about, and what is the motivation, and the novelty of their approach?

salesforce-einstein-1024x576.png

Salesforce Einstein: A virtual AI assistant embedded in Salesforce’s offering. Salesforce is looking into ways of adding explainable question answering to its capabilities.

For Salesforce Research, it’s all about question answering. This is obvious browsing through their key topics and publications. And it makes sense, considering Salesforce’s offering: would it not be much easier and productive to ask whatever it is you are interested in finding in your CRM, rather than having to go through an API, or a user interface, no matter how well-designed those may be?

Lin said:

“In the near future, we would like to enable machines to answer questions over multi-modal information, which include unstructured data such as text and images as well as structured such as knowledge graphs and web tables. This work is a step towards a building block which enables the question answering system to effectively retrieve target information from (incomplete) knowledge graph.”

She went on to add that Salesforce Research is aiming to tackle AI’s communication problem. Lin and her colleagues work on a wide range of NLP problems, spanning from advancements in text summarization to learning how to build more efficient natural language interfaces to a unified approach to language understanding:

“Deep learning is the umbrella theme of the lab, which means we also work on areas outside NLP, including core machine learning projects such as novel neural architectures and other application areas such as computer vision and speech technology.”

Not tested on real data — yet

Lin also emphasized that deep learning is not the end all. For example, it was pointed out to her that the path-finding approach Lin’s team presented which uses deep reinforcement learning is related to the “relational pathfinding” technique proposed in a 1992 paper:

“The learning algorithm in that paper is not neural-based. My take-away from this is that revisiting earlier findings in inductive logic programming and possibly combining them with deep learning approaches may result in stronger algorithms.”

The obvious point of integration would be Einstein, Salesforce’s own virtual assistant. Based on Lin’s answers, it does not look like this work has been incorporated in Einstein yet, although conceptually it seems possible. Lin explained that this work is a research prototype, using benchmark datasets publicly available to academia.

opera-snapshot-2019-03-18-162015-arxiv-org.png

An incomplete knowledge graph, where some links (edges) are not explicit.

It seems that Salesforce data and infrastructure were not used in the context of the publication. All the data Lin used could fit into a 4G RAM machine. Special data structures for representation and storage to enable fast access of the graph were not really needed, said Lïn:

“I stored facts of the graph in a plain .txt file and read the entire graph into memory when running experiments. This is the common practice of KG research in academia. To apply the model on industry scale knowledge graphs would require special infrastructure.”

Multi-hop reasoning is an effective approach for query answering (QA) over incomplete knowledge graphs. However, there are some issues with this approach: False negatives, and sensitivity to spurious paths. Lin’s work helps address those, largely by adding more links to incomplete knowledge graphs.

One thing we wondered was whether those links are stored, or generated on the fly. Lin explained that so far they have been generating answers on the fly for the prototype. But in real-world the two approaches would most likely be mixed:

“One would cache the links generated, manually verify them periodically and add the verified links back to the knowledge graph for reuse and generating new inference paths. We haven’t tested this hypothesis on real data.”

Graphs and machine learning for the win

Another contribution of Lin’s work is on what is called symbolic compositionality of knowledge graph relations in embedding approaches. Embedding is a technique widely used in machine learning, including machine learning reasoning with graphs. But this approach does not explicitly leverage logical composition rules.

For example, from the embeddings (A born_in California) & (California is_in US), (A born_in US) could be be deduced. But logical composition steps like this one are learned implicitly by knowledge graph embeddings. This means that this approach cannot offer such logical inference paths as support evidence for an answer.

Lin’s approach takes discrete graph paths as input, hence is explicitly modeling compositionality. This means it can offer the user an inference path which consists of the edges existing in the knowledge graph as support evidence. In other words, this can lead to so-called explainable AI, using the structure of the knowledge graph as supporting evidence for answers, at the expense of more computationally intensive algorithms.

knowledgegraphsmachinelearning.jpg

The combination of graphs and machine learning is a promising research direction gaining more attention as a way to bridge top-down and bottom-up AI

Combining graphs and machine learning has been getting a lot of attention lately, especially since the work published by researchers from DeepMind, Google Brain, MIT, and the University of Edinburgh. We asked Lin what her opinion on this is: Are graphs an appropriate means to feed neural networks? Lin believes this is an open question, and sees a lot of research needed in this direction:

“The combination of neural networks and graphs in NLP is fairly preliminary — most neural architectures take sequences as input, which are the simplest graphs. Even our model uses relational paths instead of relational subgraphs.”

Lin mentioned work done by researchers from USC and Microsoft [PDF], which generalizes LSTMs to model graphs. She also mentioned work done by Thomas N. Kipf from the University of Amsterdam [PDF], proposing graph convolutional networks to learn hidden node presentations which support node classification and other downstream tasks.

“It is definitely interesting to see more and more neural architectures specifically catering for which takes general graphs as input being proposed. We are seeing graphs being used to represent relations between objects across multiple AI domains these days. Graph is a powerful representation in the sense that by simply varying the definitions of nodes and edges we can model a variety of data types using it.

While inference over graphs is hard in general, it offers a potential way to integrate multimodal data (text, images, tables, etc.). UC Irvine researchers presented a really interesting paper in EMNLP, which improves knowledge graph completion by leveraging multimodal relational data. Their proposed architecture, for example, takes images and free-form texts as node features.”

The takeaway? It may be early days for graph-based machine learning reasoning, but initial results look promising. So, if one day you see your questions being answered by Einstein, along with supporting evidence for this, you will probably have graph and researchers like Lin to thank for it.

Content retrieved from: https://www.zdnet.com/article/salesforce-research-knowledge-graphs-and-machine-learning-to-power-einstein/.

Categories
knowledge connexions

Salesforce Research: Knowledge graphs and machine learning to power Einstein

Explainable AI in real life could mean Einstein not just answering your questions, but also providing justification. Advancing the state of the art in natural language processing is done on the intersection of graphs and machine learning.

A super geeky topic, which could have super important repercussions in the real world. That description could very well fit anything from cold fusion to knowledge graphs, so a bit of unpacking is in order. (Hint: it’s about Salesforce, and Salesforce is not into cold fusion as far as we know.)

If you’re into science, chances are you know arXiv.org. arXiv is a repository of electronic publication preprints for scientific papers. In other words, it’s where cutting edge research often appears first. Some months back, a publication from researchers from Salesforce appeared in arXiv, titled “Multi-Hop Knowledge Graph Reasoning with Reward Shaping.”

The paper elaborates on a technique for using knowledge graphs with machine learning; specifically, a branch of machine learning called reinforcement learning. This is something that holds great promise as a way to get the best of both worlds: Curated, top-down knowledge representation (knowledge graphs), and emergent, bottom-up pattern recognition (machine learning).

This seemingly dry topic piqued our interest for a number of reasons, not the least of which was the prospect of seeing this being applied by Salesforce. Xi Victoria Lin, research scientist at Salesforce and the paper’s primary author, was kind enough to answer our questions.

Salesforce Research: it’s all about answering questions

To start with the obvious, the fact that this paper was published says a lot in and by itself. Salesforce presumably faces the same issue everyone else is facing in staffing their research these days: the boom in the applicability of machine learning in real-world problems means there is a ongoing race to attract and retain researchers.

  
People in the research community have an ethos of sharing their accomplishments with the world by publishing in conferences and journals. That, presumably, has a lot to do with why we are seeing a number of those publications lately coming from places such as Salesforce.

The paper, presented by Lin in the 2018 Conference on Empirical Methods in Natural Language Processing (NLP), was well received. The authors have also released the source code on Github. But what is that all about, and what is the motivation, and the novelty of their approach?

salesforce-einstein-1024x576.png

Salesforce Einstein: A virtual AI assistant embedded in Salesforce’s offering. Salesforce is looking into ways of adding explainable question answering to its capabilities.

For Salesforce Research, it’s all about question answering. This is obvious browsing through their key topics and publications. And it makes sense, considering Salesforce’s offering: would it not be much easier and productive to ask whatever it is you are interested in finding in your CRM, rather than having to go through an API, or a user interface, no matter how well-designed those may be?

Lin said:

“In the near future, we would like to enable machines to answer questions over multi-modal information, which include unstructured data such as text and images as well as structured such as knowledge graphs and web tables. This work is a step towards a building block which enables the question answering system to effectively retrieve target information from (incomplete) knowledge graph.”

She went on to add that Salesforce Research is aiming to tackle AI’s communication problem. Lin and her colleagues work on a wide range of NLP problems, spanning from advancements in text summarization to learning how to build more efficient natural language interfaces to a unified approach to language understanding:

“Deep learning is the umbrella theme of the lab, which means we also work on areas outside NLP, including core machine learning projects such as novel neural architectures and other application areas such as computer vision and speech technology.”

Not tested on real data — yet

Lin also emphasized that deep learning is not the end all. For example, it was pointed out to her that the path-finding approach Lin’s team presented which uses deep reinforcement learning is related to the “relational pathfinding” technique proposed in a 1992 paper:

“The learning algorithm in that paper is not neural-based. My take-away from this is that revisiting earlier findings in inductive logic programming and possibly combining them with deep learning approaches may result in stronger algorithms.”

The obvious point of integration would be Einstein, Salesforce’s own virtual assistant. Based on Lin’s answers, it does not look like this work has been incorporated in Einstein yet, although conceptually it seems possible. Lin explained that this work is a research prototype, using benchmark datasets publicly available to academia.

opera-snapshot-2019-03-18-162015-arxiv-org.png

An incomplete knowledge graph, where some links (edges) are not explicit.

It seems that Salesforce data and infrastructure were not used in the context of the publication. All the data Lin used could fit into a 4G RAM machine. Special data structures for representation and storage to enable fast access of the graph were not really needed, said Lïn:

“I stored facts of the graph in a plain .txt file and read the entire graph into memory when running experiments. This is the common practice of KG research in academia. To apply the model on industry scale knowledge graphs would require special infrastructure.”

Multi-hop reasoning is an effective approach for query answering (QA) over incomplete knowledge graphs. However, there are some issues with this approach: False negatives, and sensitivity to spurious paths. Lin’s work helps address those, largely by adding more links to incomplete knowledge graphs.

One thing we wondered was whether those links are stored, or generated on the fly. Lin explained that so far they have been generating answers on the fly for the prototype. But in real-world the two approaches would most likely be mixed:

“One would cache the links generated, manually verify them periodically and add the verified links back to the knowledge graph for reuse and generating new inference paths. We haven’t tested this hypothesis on real data.”

Graphs and machine learning for the win

Another contribution of Lin’s work is on what is called symbolic compositionality of knowledge graph relations in embedding approaches. Embedding is a technique widely used in machine learning, including machine learning reasoning with graphs. But this approach does not explicitly leverage logical composition rules.

For example, from the embeddings (A born_in California) & (California is_in US), (A born_in US) could be be deduced. But logical composition steps like this one are learned implicitly by knowledge graph embeddings. This means that this approach cannot offer such logical inference paths as support evidence for an answer.

Lin’s approach takes discrete graph paths as input, hence is explicitly modeling compositionality. This means it can offer the user an inference path which consists of the edges existing in the knowledge graph as support evidence. In other words, this can lead to so-called explainable AI, using the structure of the knowledge graph as supporting evidence for answers, at the expense of more computationally intensive algorithms.

knowledgegraphsmachinelearning.jpg

The combination of graphs and machine learning is a promising research direction gaining more attention as a way to bridge top-down and bottom-up AI

Combining graphs and machine learning has been getting a lot of attention lately, especially since the work published by researchers from DeepMind, Google Brain, MIT, and the University of Edinburgh. We asked Lin what her opinion on this is: Are graphs an appropriate means to feed neural networks? Lin believes this is an open question, and sees a lot of research needed in this direction:

“The combination of neural networks and graphs in NLP is fairly preliminary — most neural architectures take sequences as input, which are the simplest graphs. Even our model uses relational paths instead of relational subgraphs.”

Lin mentioned work done by researchers from USC and Microsoft [PDF], which generalizes LSTMs to model graphs. She also mentioned work done by Thomas N. Kipf from the University of Amsterdam [PDF], proposing graph convolutional networks to learn hidden node presentations which support node classification and other downstream tasks.

“It is definitely interesting to see more and more neural architectures specifically catering for which takes general graphs as input being proposed. We are seeing graphs being used to represent relations between objects across multiple AI domains these days. Graph is a powerful representation in the sense that by simply varying the definitions of nodes and edges we can model a variety of data types using it.

While inference over graphs is hard in general, it offers a potential way to integrate multimodal data (text, images, tables, etc.). UC Irvine researchers presented a really interesting paper in EMNLP, which improves knowledge graph completion by leveraging multimodal relational data. Their proposed architecture, for example, takes images and free-form texts as node features.”

The takeaway? It may be early days for graph-based machine learning reasoning, but initial results look promising. So, if one day you see your questions being answered by Einstein, along with supporting evidence for this, you will probably have graph and researchers like Lin to thank for it.

Content retrieved from: https://www.zdnet.com/article/salesforce-research-knowledge-graphs-and-machine-learning-to-power-einstein/.