First published on Medium.com.
In business, vision isn’t some mythical ability to see the future. It’s about being able to recognize a pattern and apply it to something new, before others see it coming.
In this post, we’ll introduce you to one such pattern, the gestation of new media within old media. We’ll then review some examples of how the pattern has repeated itself over the past 30-years, from one Big Thing to the Next.
We’ll then apply the pattern in the here-and-now, to see how it points to The Next Big Things. Read more …
Joyce Hostyn argues that Better Human Understanding, Not Big Data, Is the Future of Business. Some excerpts (with my emphasis):
Despite the best of intentions, we’re not data driven, we’re hypothesis driven. Our stories (our mental models) are merely hypotheses of how the world works. But we see them as reality and they influence what data we collect, how we collect it and the meaning we glean from it…
In a quest to become data driven, are marketers trapping themselves with outdated mental models of data and analytics? “Big data is being wasted on marketing. The true power of analytics is in revealing cultural dynamics.”
Hostyn concludes her thoughtful article with a number of questions, including:
- Can we leverage big data to zoom out and understand patterns and trends, then zoom back in for a dive deep into the hearts and minds of individuals?
- Are we willing to develop hypotheses with the potential to disrupt our old mental models? Create experiments to test those hypotheses. Prototype to think. Collect feedback. Iterate.
At Primal, we’ve invested years exploring this mode of hypothesis-setting as a lens into big data. It involves a collaboration between humans and machines across the full spectrum of analytical and synthetical thinking.
What follows is a summary of that exploration and what we’ve learned to this point.
By Tony Sarris
Call me sentimental, but perhaps some of you, too, are starting to miss the human side of sentiment analysis.
Machine-based statistical analysis is not the entirety of sentiment analysis – or at least it shouldn’t be. There is still an important role for humans to play in that process. In addition there are opportunities for other tools to assist humans in the process, working in complementary ways with statistical-based sentiment analysis tools. Read more …
Developers have long created software that customers use directly. But now, we’re creating solutions that incorporate internal representations of end-users and adapt to individual needs.
In this post, we’re going to introduce you to the most important component in your solution stack, the user model, explain why it’s so important, and show you how to incorporate user modeling into your solution.
By Mike Rolfe
Last week marked Primal’s first release for the month of August. Over the past month, we’ve been busy updating our Data API, as well as taking steps to simplify our developer sign up process on the Web and through our new WordPress Plugin.
An update to Primal’s Data API exposes more of our underlying technology to our customers by way of the “Concept Confidence” score in the JSON response. This score is intended to give you more insight into the expected quality of Primal’s data.
Additionally, we completed improvements to the content quality available through our News source.
Read more …
By Norm Malloch
Justin Kominar is a Senior Software Developer at Primal
Recently, Justin Kominar, one of Primal’s developers was interviewed by 3scale. Primal’s uses 3scale’s product to manage access to our cloud-based Data API. There was so much interesting developer stuff in the interview, we thought we re-publish it in it’s entirety! Give it a read as Justin walks through some background details on how Primal’s user modelling technology is built, tells how we find interesting content for your customers, and shares some insight into new software agent technology we are working on.
Source Talking Politics
Bots have a bad reputation. Whether they’re sinister botnets, carrying out coordinated attacks, or annoying spambots, polluting our digital universe with crappy auto-generated content, bots are a much maligned bunch.
You might ask, if bots are bad and a huge portion of Twitter activity is already generated by bots, Why is Primal creating another one?
We believe that intelligent, well-intentioned bots can add a lot of value.
If we build a bot that truly understands you as an individual and can do useful work on your behalf, it might be a welcome addition to the social scene.
Read more …
By Derek Wyatt
We’re using Spray for some products here at Primal. Spray is a really neat library, but it’s missing the ability to gather runtime metrics, and while you can instrument your app any way you’d like, we want to do things the Spray way.
Along with our upcoming Agent framework, which implements a pretty cool inter-actor protocol, we have other services that will be hosted in Spray. The guys that figure out whether or not people like our features, and the other guys that figure out how well they’re working, want to know what’s happening, and the guys that code it don’t want that stuff interfering too much with the way that they write their code.
In this post we’ll have a look at how we’re integrating the Coda Hale Metrics library with Spray. Eventually we’ll be making a pull request to the Spray project to integrate this thing for real.
Our surprising conclusion: Marketers and technology buyers are sold on the promise of personalized recommendations. Unfortunately, they don’t know how to tell recommenders apart, even when they’re built on radically different technical approaches. And the results are mixed, at best. End users are left frustrated, wondering when this promise of personalized recommendations will actually be delivered.
In this post, we’re going to show you how to ask the right questions of your technology provider, to make an informed choice based on business considerations, not technical jargon. We’ll highlight some of the common risks and pitfalls, and conclude with a statement of what to expect from your technology provider. Read more …
Yesterday marked the release of our last major product update for the month of June.
For our data API, improvements were made to our scoring, the quality of the abstracts that we return, and a big upgrade in both the quantity and quality of the content sources that we monitor in our default News content source.
Read more …
By Mike Rolfe
Small conversational data such as tweets or text messages are a goldmine of individual interests. Millions of people everyday tweet about their favourite food, send a Kik or BBM message about a recent TV episode, or take a photo on Instagram while attending a sporting event or concert.
However if you’re trying to recommend content or promote offers within a conversation, the ability to determine a user’s interests or “commercial intent” from these small data inputs is a huge challenge.
John Koestier in Venture Beat writes:
“…tweets are difficult to register commercial intent…
If I tweet about my wife’s illness, are you going to target me with a random medicine? Or if you tweet about a great dinner you’re just about to eat, will you really be receptive to ads about a Greek restaurant just down the road?“
In this post, we’ll show you an elegant, “small data” solution, applied in one of big data’s harshest environments.
Read more …
Recently, the idea of a portal is getting a reboot. But as much as the Web has advanced for human consumption, it remains a very inhospitable place for a large contingent of users, namely machines.
Back in the early days of the Web, the Yahoo! Directory filled a huge gap in the ecosystem.
This portal helped us make sense of the Web. It organized content based on topics, connected those topics into a hierarchy to make sense of it all, and standardized the presentation of the content so that it was convenient and accessible.
But where can you find a portal that makes content from the Web accessible in your application?
By Mike Rolfe
Do you want to provide highly relevant content from around the Web and engage your readership on a whole new level?
Primal for WordPress will allow you to unlock the power of your WordPress site’s interest graph to supply your readers with relevant, dynamic and daily updated content that is tailored to each individual page.
This first revision of our plugin has all of the functionality necessary to speak to Primal’s data service directly, however we’d love to hear from WordPress publishers and developers on exactly how else you would like to leverage the power of the interest graph within WordPress environment as we consider where to take the plugin next.
We know what personalization means and the compromises it imposes on our individual privacy.
Or do we?
This is perhaps the most insidious myth among the technorati: In order for people to benefit from advanced and personalized technologies, they need to compromise their individual privacy.
This idea is remarkably pervasive and damaging, driving both consumers and businesses away from the opportunities of personalization and next-generation information services.
In this post, I’m going to introduce you to the myth and the underlying villain, Big Data. I’m also going to argue that innovation is a much better path forward than evil, or doing nothing at all.
Read more …
Our newly redesigned developer site makes it easier than ever to explore the Primal data service and harness the power of Primal in your own applications.
In minutes, you can be up and running, trying out the standard HTTP calls that you use to talk to the Primal data service. Soon you’ll be building your users’ interest graphs, retrieving information from them, and filtering content that matters to you and them through their interest graphs.
Read more …
Primal and the world of interest networking, for the month of May. Product updates, highlights from our blog, and industry news.
By Derek Wyatt
We’re building some cool new stuff at Primal. Awesome ideas and talented people are a must have in order to build successful and exciting products, but we also need our tools to step up to the challenge. This is why we’ve started adopting Scala and Akka. In this post, I’m going to describe how we’re using Scala’s implicits in order to implement a very important part of our internal messaging fabric, without having to over-burden the business logic. Read more …
“Innovation will disrupt many areas of skilled work that have so far had it easy. But if we manage them well, smart machines will free us, not enslave us.”
- The Economist (May 25 2013)
The moral of the story: Either you use smart machines to your competitive advantage, or you concede that advantage to your competitors.
Primal is one such smart machine. It’s intended to empower small- and medium-sized businesses by being simple to use and affordable.
In this post, we’ll show you how to add Primal’s data service to your content curation solutions, as a fully automated, machine-editor.
We’ll also highlight the practical applications, concrete benefits and costs of using a smart machine as a complement to your manual and crowdsourcing strategies. Specifically:
1. How to filter out irrelevant content from your content supply.
2. How to provide personalized collections of content.
For our demo, we’ll use a content aggregator called Alltop, and show you how to recreate these examples and build a similar solution yourself!
I was meeting with two guys, one a technologist, one a business advisor.
The discussion was focused on Primal’s technology: semantic user models, knowledge representation, yada, yada…
The business advisor, having listened patiently for some time, finally interjects, “Tell me what this means to Trixie!”
“Yes, the everyday person. Tell me a story of why Trixie would care about any of this?”
Read more …
By Derek Wyatt
Primal does a lot of heavy lifting in knowledge representation and content filtering. If you ask it to grab you some relevant content around your interests, it will do precisely that.
But what if you don’t want to have to ask? Search engines are fantastic, but they still require that you go to them and then try to figure out how to formulate your query in a way that gets you decent results.
Primal already has the ability to understand what you want, and we’re now working on some technology that will let Primal deliver you the content that you truly care about before you know you want it.
I love this post from Fogbeam Labs. Here’s a bit:
So given that, what can we say about the eventual development of something we can call “The Star Trek Computer”? Right now, I’d say that we can say at least two things: It will be Open Source , and licensed under the Apache Software License v2. There’s a good chance it will also be a project hosted by the Apache Software Foundation.
Their rationale? ASF provides an awesome array of advanced technologies, in everything ranging from NLP, information extraction and retrieval, machine learning, Semantic Web, and on and on. It’s like a free, all-you-can-eat buffet! (er, Star Trek food synthesizer?)
I share their enthusiasm. We use many of these technologies at Primal.
But this is where their science fiction story starts to lose me:
Of course, you don’t necessarily need a full-fledged “Star Trek Computer” to derive value from these technologies. You can begin utilizing Semantic Web tech, Natural Language Processing, scalable machine Learning, and other advanced computing techniques to derive business value today.
We often meet product developers and entrepreneurs looking to build next-generation intelligent solutions.
If these advanced technologies are available for free, why not just jump in and start building?
Read more …
Everyone’s talking about the Yahoo-Tumblr acquisition, specifically, whether Yahoo will “mess up” Tumblr.
Here’s what Mayer said about it, as reported by TechCrunch:
“Our strategy is to let Tumblr be Tumblr,” said Mayer. “There are some who will always prefer Tumblr and will never come to Yahoo. [But] as we pull Tumblr content into our news feed and media experiences it will cause them to become that much more interesting and richer and will cause more to come to Yahoo.”
I think the much more interesting question is, How will Yahoo integrate Tumblr content? It’s a massive technical undertaking.
At Primal, we love working on interesting problems, so here’s a peek into what a Yahoo-Tumblr experience might look like, and why it’s such a daunting technical challenge.
Content and media companies are getting excited about personalization (again). Companies like Gravity are touting the benefits: content that’s targeted to individual users generates more revenue for publishers and a more tailored experience for consumers.
Perhaps you want to join the personalization party, but you’re skeptical. What’s the catch?
If you’re like us, you find the notion of tracking and analyzing consumer behaviour offensive. You want to treat your consumers like individuals, but you don’t want to creep them out or betray their trust.
Most importantly, you want to bring your own expertise and knowledge to bear. You don’t want to categorize your consumers based on someone else’s knowledge. You want to use your perspective and interests as their guide.
Here’s how you can do-it-yourself with Primal.
Read more …
Imagine this: I found a book in the bookstore that was almost what I was looking for. So I walked up to the store clerk for some help…
“Do you have any other books like this one?”
“I don’t know. Would you like to see other books that other people who looked at this book looked at?”
“No, that sounds silly (and a bit creepy). I’d like to see more books like this one.”
“OK. Would you like to see one of the hundreds of books that are in the same category as this book?”
“No, that sounds like a lot of work. I’d like to see more books like this one.”
“OK. Would you like to review any of the recommendations based on our surveillance of your browsing habits over your many visits to our store?”
“No, that’s completely creepy!! I’m outta here…”
We don’t have this sort of experience when we interact with real people, but this is what content and product recommendation engines subject us to every day.
How do you recommend products or content for a very specific interest at a moment in time?
In the sections that follow, we’ll show you how to build a working recommendation engine using Primal’s data service, AlchemyAPI, and some code samples to tie the pieces together.
Read more …
I love this post by Joshua Fruhlinger, Please don’t personalize me. I know who I am. It captures a sentiment that many of us share (1) (2) (3). Joshua provides a wonderfully non-technical and simple summary:
I know who I am. I don’t need Facebook or Google or Microsoft or Apple or anyone else to collect data and tell me what I’m interested in. I’m pretty sure I know what I like and don’t like.
It’s a great point: Not only are big data approaches to personalization privacy-invading and offensive, they don’t work very well!
There are much more direct and transparent ways to approach the opportunity of personalization. A system that allows you to be the master of your interests is self-empowering. In this post, I want to tackle one of the most common questions we receive: How do I train Primal to represent the interests I care about? 1
Some things are too important to leave to anyone else. For me, the health of my family is one example. There isn’t an aspect of that interest that I can live without. And when new information surfaces, it needs to find me.
Whether its your health, your business, or your life’s purpose, you need comprehensive information and a service that’s dedicated to your individual interests, 24/7.
Where can you find an information service dedicated to the topics and content you truly care about? Your interest network would be as unique and one-of-a-kind as you are. But you don’t have the time to build it yourself and you don’t have the money to get others to do it for you.
Welcome to Primal, your interest network, made-to-order. It’s free for individuals and affordable for companies of all sizes. And now anyone can start an interest network. Read more …
Where are all the Flipboard killers?
(Or how to build a personalized news app with only 10-lines of code.)
Calling all innovators: we’re looking for people who can imagine a legitimately new experience for personalized news and information services.
Personalized news and information services have become common. What’s uncommon is a service that actually delivers a truly individualized experience; something that really moves the category forward.
In the sections that follow, we’ll show you how Primal can support your product development efforts by providing the data that describes the interests of individuals.
And we’ll show you how to do it using only a few lines of code.
By Ken Bryson
As part of their 15th anniversary celebrations, Communitech decided to check in with a few key entrepreneurs from Waterloo Region’s world-class technology community, including our own CEO Yvan Couture.
In this video series, Yvan reflects on the genesis of Communitech and his role in growing Waterloo Region’s technology community.
By Ken Bryson
Primal connects your personal interests to related topics and content from across the real-time Web. Primal makes it easy to put your interests to work by following and sharing the information you truly care about!
Launched today, Primal’s Web app is your new entry point to interest networking.
We are constantly exploring interesting topics from across the real-time Web. You can explore Primal’s interest network simply by clicking on a topic that interests you. From there, you’ll find dozens of connected topics of interest and content items to explore, such as news, videos, social media, and images.
The most frequently asked question about Primal is, Where do you get the interests data? It’s a fair question. If Primal is the most comprehensive source of open interests data in the world, it begs the question of where we get this valuable data. (1)
Unlike our competitors, we don’t ingest and analyze vast amounts of historical consumer data to derive interests data. Instead, Primal’s computational engine creates this data on a just-in-time basis, with each and every request.
The reason Primal seems to have data about every conceivable topic of interest is that Primal is literally creating the data on-the-fly. Read more …
By Yvan Couture
You can’t do it alone! Starting and growing a company requires dedicated founders, strong internal leadership, and exceptional employees. But it doesn’t stop there. Having outside advisors and external board members brings vital experience, fresh perspectives, and guidance that’s critical to success.
At our Annual General Meeting in June, our shareholders re-elected our lead investor and external board member, Jim Estill, and welcomed three new external board members to our team: Dr. Savvas Chamberlain, Dr. Larry Cornett, and Dr. Paul Hofmann. They join our founder, Peter Sweeney, and myself on our Board of Directors. Read more …
Learning Sherpa, a Primal-powered application that tailors educational content based on the intersection of interests between educators and students, was awarded The Learners’ Choice Prize in the Desire2Learn Edge Challenge. Learning Sherpa was selected as the submission with the most potential to have a lasting impact on teaching and learning.
We’re less than 2-months into alpha testing with developers, but already we’re discovering some surprising ways to use Primal:
Interest graphs simulating human activity
Imagine filling out your social network with digital companions or forming focus groups of imaginary people for your market research.
Interest graphs are supposed to represent the interests of real people, right? But what if real people aren’t available? Primal can create interest graphs using any set of interests for each persona. Let these digital personas loose on the Web to explore content and even interact with real people!
The fervor around big data continues to grow. The World Economic Forum and The New York Times are jumping on the bandwagon. While we share their enthusiasm for the potential, big data needs a reality check.
Here are just a few of the how-do-you-get-there-from-here questions for anyone considering big data projects. Read more …
By Tom Levesque
A growing list of companies are making “interests” the focus of their value proposition: Twitter allows you to “follow your interests”; Gravity “unlocks the interest graph”; Pinterest “organize(s) and share(s) the things you love”; Quora “connects you to everything you want to know about”; Chime.in “connect(s) around your interests” — just to name a few. Read more …
By Tom Levesque
At Primal, we believe software agents have a big role to play in the Web’s evolution. More like virtual assistants than applications, software agents work continuously on your behalf, delivering value even while you’re away from your computer or smartphone.
But a virtual assistant is only as smart as its data. Imagine you commissioned an assignment like, “keep an eye on the latest news, and email me when you find a story I will like.” How could you communicate your specific interests to your virtual assistant? Read more …
Primal’s Mo Bros for 2011.
During November each year, Movember is responsible for the sprouting of moustaches on thousands of men’s faces around the world. With their “Mo’s”, these men raise vital funds and awareness for men’s health, specifically prostate cancer.
If you’d like to help out, please donate.
By Tom Levesque
Excerpt from The Waterloo Region Record, Technology Spotlight 2011
By Chuck Howitt, The Record
Two years ago Tony Sarris was living the American dream. He was an engineering director for Unisys, a large U.S. information technology company with 37,000 employees worldwide. He lived in the west coast paradise of Laguna Beach, California. He made a comfortable salary.
Today, he works for a small Waterloo internet search company with 35 employees that few people have heard of. Goodbye surf, sand and sun. Hello grey skies, snow and cold winters. Has he lost his mind?
By Tom Levesque
Have you ever pondered how semantic engines, vertical search, and interest networking relate to enterprise mobility? In this podcast, Kevin Benedict interviews Primal’s founder and co-president Peter Sweeney on these subjects.
Big data technologies are plagued with small data problems. Their performance suffers in markets that aggregate a large number of unique interests. Some of the largest markets share these small data characteristics, including local ecommerce, personalized media, and interest networking. New approaches are needed that are far less sensitive to the cost and complexity of the data.
By Tom Levesque
In the early days of the Web, librarians would often compile directories of “trusted sites” on a range of important topics. In fact, Yahoo! has its roots in what was essentially a human-edited directory of online content — then called David and Jerry’s Guide to the World Wide Web. These are early examples of online content curation.
A content curator is someone who finds, groups, organizes and shares the best and most relevant content on a specific issue. In the past, content curation was largely the domain of small groups of professionals.
Today, services like Tumblr have begun to attract part-time, amateur content curators by making it easier than ever to “publish” the interesting things you find online. While these services have undoubtedly brought content curation to new audiences, the time and effort involved remain significant barriers.
Further technological innovation is needed to lighten the burden of the entire process of finding, organizing, and grouping, and sharing content.
Taking the work out of content curation
One problem with existing tools is that they force people to work at the level of individual pieces of content. Given the flood of content on the Web, this approach requires a lot of manual labor on the part of a content curator. If the manual approach is the only one available, content curation is doomed to be a niche activity.
A new system is needed — one that puts raw computing horsepower in the hands of content curators to help them get the job done orders of magnitude faster than they can today.
Content curators want to spend their time focusing on their editorial vision, pulling together the ideas that matter most to them and their audience. They likely don’t want to wade through hundreds of pages of search results and feeds, painstakingly organizing what they find.
The key, then, is to enable content curators to quickly express their ideas, indicate a few sources to get content for those ideas, and let computers do the hard work of finding and organizing content around those ideas. The final step is for the content curator to vet the work of the computer, modifying the results as they see fit to give the results that “human” touch.
With a product like this, content curation could become easy enough — and fast enough — that anyone could do it.
Using Primal to do content curation
Primal’s technology for understanding individual interests and Web scale content filtering takes much of the grunt work out of content curation.
Primal, a pioneer of Internet automation technology, today announced that Larry Cornett has joined the company’s board of advisors. Cornett is the founder and CEO of Brilliant FORGE, a product and design strategy and management consulting firm focused on Fortune 500 companies and startups. Recently, Larry was the Vice President of Consumer Products for Yahoo! Search. For the past 16 years, Cornett’s career has focused on designing, defining, and building consumer products at several of the biggest and best technology companies, including Apple, eBay, and IBM. Read more …
By Tom Levesque
Technology was supposed to revolutionize our lives. There were promises of 20-hour work weeks, robotic servants to do our bidding, and leisurely weekday afternoons in the sun. That was a fantastic dream. So what happened along the way?
Today, we face the grim reality that most of the technology we build simply enables people to do more work.
Your PC is perhaps the best example of this. Sure, it’s a powerful tool. But it’s one that can do almost nothing without a human driving it. You respond to your emails. You browse the Web. You write that report. And you fix it when it breaks.
Could a computer do some of that work for you? Read more …
Industrialization is transforming our information economy, destroying old business models and creating new opportunities. The impact it will have on content professionals will make Web 2.0 seem tame in comparison. To understand this transformation and leverage it effectively, you need to parse the myths from reality.
I’ve argued that industrialization is the most transformative force on the Internet. Human tasks are rapidly being displaced by machines. Factories of advanced technologies are being constructed to automate the manufacture of information and content. Predictably, there is much hand-wringing and righteous indignation expressed about this economic sea change. Read more …
How Do We Roll Out The Semantic Web? Paradoxically, the fast track may involve getting help from billions of people who know nothing about the Semantic Web and have no interest in it.
Challenges with current approaches
Most of the current approaches to building the Semantic Web focus on content. We create semantic representations of existing assets such as databases, documents, and social media. Machines “read” this knowledge and execute tasks on behalf of consumers. In the Semantic Web world, the approach is content first, consumers second.
Unfortunately, semantifying content is proving to be an extraordinarily daunting task. When we expand the scope of the problem beyond our existing content assets to include knowledge generally, in all its subjective and boundless glory, the challenges of a content-first approach becomes clear. We need alternative strategies, and more importantly, many hands on the problem. Read more …
Web 2.0 is social: many hands make light work. In stark contrast, Web 3.0 is industrial: the automation of tasks displaces human work. But trite definitions won’t prepare us for change. Whatever you call it, our information economy is in the midst of an Industrial Revolution. And if you don’t place the Web within the frame of industrial manufacturing, you won’t see the real disruptive change coming.
This story reads much like the first Industrial Revolution. Artisans and skilled tradesman used to create everything by hand. Then, through the emergence of a handful of technical innovations, came the age of mass production. It was a profound turning point in human history, affecting every aspect of daily life.
Today, most content is still created by hand, the best of it by highly skilled artisans drawing on centuries of scholarship and experience. Recently, we’ve seen significant innovations in social approaches to content creation. But Web 3.0 industrialization takes content manufacturing to an entirely different level. Instead of users manually creating content, machines automate the heavy lifting. Consumers simply push the buttons and get stuff done. Think spinning wheels versus textile mills.