Analytics

Exploring iHub Examples: SF Wealth

This post is the third in a series exploring the free example applications that come bundled with OpenText Information Hub (iHub), Trial Edition. Read part 1 and part 2. Financial services firms differentiate themselves and build loyalty by providing their customers with in-depth portals for exploring personalized financial data. Many top banks and other providers use OpenText Analytics and Reporting tools to present, analyze, and report on customer data, and the iHub example application we explore in this post showcases just a few of the capabilities at their disposal. The example application is called SF Wealth. It’s a dashboard with five tabs – Home, My Spending History, My Wealth, Retirement Roadmap, and Portfolio Performance – each of which illustrates different capabilities. When you launch iHub, Trial Edition and open the Examples, click on the SF Wealth image and you’ll see the screen below. Home What it is: A landing page – that is, the first screen a high net worth customer might see when logging into a financial institution website. What to look for: The page presents personalized information about the customer’s portfolio status, progress toward goals, and risk position in snapshot form. Down either side you’ll see links to outside resources of interest to our imaginary customer. In a real-world portal these HTML links could point to offers geared toward the customer’s specific needs and wants, news headlines about companies in the customer’s portfolio, and other relevant information. We’ve included them here to show how seamlessly third-party, outside sources can be incorporated with data that is unique to the customer. My Spending History What it is: A dashboard (shown above) that lets a customer explore spending in depth. What to look for: To see how various aspects of the same data can be presented in different ways, first select the Restaurant category (in the left column) and watch chart immediately to its right; it will show spending on restaurants, color-coded by account. Now Clear the Categories, find Restaurant in the bar chart, and click its bar; you’ll now see details about all of the restaurants where our user spent money. You can also experiment with the Month Range in the selectors and see how all four graphs change. My Wealth What it is: A dashboard (shown above) that gives the user multiple ways to gauge progress and performance. What to look for: The seven elements on this dashboard mostly provide a variety of comparisons. The thermometer at left shows that our user has saved almost $968,000 – a good figure, until you compare it to the user’s $2 million goal. Now check the map in the lower right corner; our user’s savings are above average for Idaho, but below average for Illinois. (Maybe his money would go further – or he’d feel richer – if he moved.) Two of the gadgets on this dashboard are interactive. My Account Update lets you get a detailed report on any fund by clicking its name, and My Wealth vs. Market Indices lets you adjust the timeframe of the chart with a click. Lastly – and just for fun – check the world map in the lower center of the dashboard; our imaginary user has travel goals and is measuring a travel fund against those goals. Retirement Roadmap What it is: An interactive page (shown above) that lets the user experiment with a variety of retirement savings scenarios. What to look for: This dashboard combines two data visualizations, a selector, and a crosstab, all of which interact with each other. The quickest way to see this in action is to enter a retirement year and choose an investment style – Conservative, Moderate, or Aggressive – and watch the all three elements change. If the user decides that a different investment strategy is needed, a financial advisor is just a click away using the Contact Us button.  (Note that this example application uses artificial data. Please don’t use it to plan your own retirement!) Portfolio Performance What it is: A page that packs a ton of financial performance information into an efficient table. What to look for: The table (shown above) is a marvel of efficiency and demonstrates iHub’s powerful capabilities. Each line in the table includes five charts in three styles, along with some of the numbers that underlie those charts. You’ll also see scorecard-style red dots indicating a portfolio item that is – appropriately enough – in the red. With this table you can quickly identify which portfolio items are performing well year-over-year, and which items are riskier than others. One More Thing If you’re following along in the Example application, you probably noticed that we’ve cropped a toolbar out of the screenshots in this post. (It’s shown above.) You also may have noticed the light blue grid behind the pages. Here’s what those things tell you: You’ve actually been working in the visual dashboard design tool that is built into iHub, Trial Edition. Because that’s so, you can rearrange and resize many of the elements on these dashboards. (Try it!) We also encourage you to create a new tab in the example application and experiment with these tools. You’ll want to keep the iHub Dashboards documentation and tutorials handy as you do. Next Up In our next blog post in this series, we will explore another dashboard – the Call Center example application – from the user perspective.  

Read More

Data Driven Digest for September 25: US Map Potpourri

In his book, How the States Got Their Shapes, author Mark Stein notes that many of the lines that separate us are arbitrary at best; based on historical negotiations and treaties with other countries. It’s not uncommon for communities to be split down the middle of two states just because an 18 Century surveyor identified a geo-spatial line. Do the lines separate us or do our behaviors? We ask this because this week’s Data Driven Digest focuses on three maps of the United States that are distinct more because of the behaviors of the people that live in them than the actual lines that separate them. Enjoy! You Want Fries With That?: If you are hungry for data, our friends over at DataBucket serve up a hot dish of information. An interactive map based around prices at Five Guys Burger and Fries restaurants tracks prices on four items available at the restaurant: Bacon Burger, Fries, Bacon Cheeseburger and Hot Dog. It’s no surprise that a meal in Midtown Manhattan costs nearly $3.00 more than one in Kansas City, Kansas. The map is novel, however, in that it displays data gathered by using Five Guys online ordering website. The 29-year-old chain currently has 1,000 locations in the United States, Canada, Britain and United Arab Emirates with 1,500 more under development. There is one item available at the restaurant that is priced consistently. All roasted in-shell peanuts found in the dining area are free. No Debate: With several months to go before the 2016 US Presidential election, we’re taking a look at some early returns. Washington Post writer Philip Bump (@pbump) notes that the upcoming televised Presidential debate in St. Louis, Missouri will be the fourth such time the city has hosted a presidential or vice presidential contest in the last 56 years. Bump suggests the St. Louis bias may come from the familiarity with host Washington University of St. Louis as well as Missouri’s past swing-state status. The Post’s visualization’s auto-build is quite impressive. You can clearly see the 28 states that have yet to host the marquee stump. Three other sites slated for next year include Dayton’s Wright State University; Farmville, Va.’s, Longwood University; and University of Nevada, Las Vegas. Startup Funding Nation: Silicon Valley startups get a lot of attention from venture capital firms, but how much is much? Our friends over at DataBucket used datasets from CrunchBase drawn from Github and built it into an interactive map that tracks the average funding amount and number of companies for each state and funding round. The wealth distribution weighs so heavily in California that the second largest state for funding (New York) is nearly a quarter of the investments flowing into the Golden State. While any state can spawn startup seed money (yes, we are looking at you South Dakota) emerging venture capital hotspots include Florida, Texas and Massachusetts. Startups are big money. VCs invested more than $48.3 billion in 2014 according to data released by the National Venture Capital Association. That seed money supported upwards of 4,356 deals. Series A funds are typically used to get a company going, while successive funds are used to help grow the business. After receiving $1.6 billion in early funds, Uber is currently locking into place a $2.8 billion to expand. Elon Musk’s SpaceX grabbed a $1 billion-dollar cash infusion valued at a more than $10 billion. Recent Data Driven Digests: September 18: Original US Congressional debt, Euro spending, and world population based on income September 11: Cloud visualizations related wind, bird migration and El Niño September 4: Seasons represented by fall color, energy production, wildfire smoke, air pollution  

Read More

Exploring iHub Examples: Integration Framework

This post is the second in a series exploring the free example applications that come bundled with OpenText Information Hub (iHub), Trial Edition. Read the first post here. The ability to integrate and embed data within other applications is an iHub strength. This ability is particularly interesting to independent software vendors (ISVs) who want to embed analytics in their applications, and it’s thoroughly demonstrated in the Integration Framework example application. Once you launch iHub, Trial Edition and open the Examples, click on the Integration Framework image and you’ll see the Mashboard screen below. (More about that name in a bit.) Mashboard What it is: The first screen you get when entering the Integration Framework example application is a modern tile-style page – one with a few surprises up its sleeve. What to look for: Some attractive visuals are immediately apparent, such as the subtle animation when you mouse over the four boxes at the top of the page. But to ISVs, more important is the way the page is constructed: the charts, maps and tables are created with iHub, but the boxes that contain them are created by the application itself. ­(That’s where the name Mashboard comes from – the page is a cross between a mashup and a dashboard.) All of the visualizations on the page are interactive, and the minus/plus icons in the upper right of each tile in the dashboard let you minimize or maximize the individual visualization independently of the others. To explore the other Integration examples, pull down the menu icon at the top left corner of the page next to the words Sample Application. You’ll see five choices: Reporting, Analytics, Web Tools, Charts, and Embedded Code. Select Reporting. Reporting What it is: Three different reports – Sales Order, Worldwide Sales, and On-Demand – that show iHub’s ability to generate and integrate reports. What to look for: Each of these reports illustrates different strengths and integration capabilities, so let’s look at them individually. Sales Order Report (shown above) combines infographic-style presentation of key metrics, a comparison bar chart, and two interactive tables of data. The lower right table uses an inline bar chart for each record (in the Actual vs Target column) as well as a red/yellow/green scorecard presentation for numbers. Click on any salesperson’s name and you’re taken to a dashboard for that individual. The first page of this report presents aggregated data, and subsequent pages give detailed reports. (Page navigation is found in the upper-right corner of the screen; it’s cropped out of the screenshot above.) Go to one of the subsequent pages, Enable Interactivity (as explained in the previous Examples post) and you’ll see that these aren’t ordinary static reports. You can easily sort data in the tables, hide and rearrange columns, and otherwise fine-tune the reports’ appearance and performance to suit your needs. Worldwide Sales (shown above) is a report organized as a grouped table. Each page of the 20-page report presents data for a different geographic area; note how the infographic header changes as you click through the pages. The Order Status column uses a scorecard-style color code, so the type color changes depending on the displayed text. On-Demand Report, when you first select it from the menu, appears as a nearly blank page. Choose an item using the pull-down Parameter Picker in the upper right corner, click Run Report, and iHub immediately generates a detailed invoice for the customer, like the one shown above. (This is not a static page, but rather is a generated report.) Incidentally, iHub uses its JavaScript API (JSAPI) to display the parameter picker; significantly for ISVs, the JSAPI parameter module and iHub’s viewer module are integrated seamlessly on a single page. Analytics What it is: A dashboard with four tabs – Inventory & Sales, Cross Tab, Performance, and Other gadgets. What to look for: Each tab shows a different way that data can be integrated and displayed in a dashboard. We’ll consider each option individually. Inventory & Sales dashboard showcases iHub’s treeview control. Explore the selectors along the left side of the dashboard; you can make selections, click Apply, and watch the graphs change. You can select any combination of whole countries and cities within countries; when you do, notice that the additional selectors at the top of the page (Cancelled, Disputed, etc.) also change according to the available data. Hovering over elements in the charts causes detailed figures to pop up, as seen above. Cross Tab demonstrates that you can build a dashboard that contains a cross tab. (It sounds simple – and with iHub it is – but not all dashboard-creation products have this capability.) Selectors on the left side of the dashboard enable you to filter the data that is presented in the cross tab Performance shows a variety of simple meters and thermometers, based on data picked using the selectors at the left of the dashboard. Don’t like the appearance of a meter? Click on the meter, click the triangle on its upper right corner, and select Change Type to see what options are available. (Developers control what other chart types are made available to users when the dashboard is created.) Other Gadgets shows how you can embed a live web page – in this case, our own developer site — into an iHub report. This capability exists because many business dashboards also must provide portals to external data and content. Web Tools What it is: A direct link to iHub tools for building things: Dashboard, Interactive Crosstab, Report Studio, and My Documents. What to look for: The four options here allow you to experiment with the visual developer tools that come with iHub. There’s too much here to highlight in a blog post, so keep the iHub documentation and tutorials handy as you explore these examples on your own. But one thing you can easily do, even without reading the manual, is to compare the user interfaces of the Dashboard builder and Report Studio. You can also check out the many types of data visualizations that come with iHub, shown above. (We’ll talk more about your data visualization options in the next section.) Charts What it is: A collection of data visualization examples, both native to iHub and built using third-party technology. What to look for:  These visualizations are grouped into four broad categories: Lines – Columns/Bars, Pies – Donuts, Other Visualizations, and 3rd-Party Visualizations. On the Lines – Columns/Bars page, note that the three visualizations across the top show the same data in different formats. This illustrates the fact that choosing the correct visualization for a given situation is not always easy. Same goes for the Pies – Donuts page; the same data is presented in a number of different ways. To learn more about choosing the right data visualization for your needs, see 8 Tips to Big Data Visualization and UX Design, featuring a video of our own Mark Gamble discussing data visualization best practices. The Other Visualizations page shows six other data visualizations that come out-of-the-box with iHub, and the 3rd-Party Visualizations page presents just a few of the JavaScript-based visualizations that iHub can bind data to and render. Your options here are almost limitless; learn more about using iHub Custom Visualizations here. Embedded Code What it is: Three examples of how you can add code to iHub content to increase the content’s functionality and improve its appearance. What to look for: The first Embedded Code item, JQuery, shows how a few lines of JQuery code can be used to enhance standard iHub tables. The left table has extra highlighting (here’s a blog post explaining how that’s done), and the right table demonstrates one-click expand and collapse capability. In both of these cases, the report tables were developed using iHub’s standard table tools, and JQuery code was applied after the fact to provide a different interactive experience. The second Embedded Code option, GoogleMap, shows how a familiar map format can be embedded and integrated into an iHub application to provide location intelligence. The quickest way to see how data from iHub affects the map is to right-click the State column, choose Filter, and then set the Condition as Equal To CA. The map will zoom in on California and only show entries for that state. The final Embedded Code item, Custom ToolBar, changes the Sales Order Report that we explored earlier by replacing the standard menu pull-down and the page navigation selectors with versions that better match the report’s overall style. Again, this is accomplished with a few lines of code, and shows how seamlessly iHub visualization and reporting can integrate and blend with your own content. Next Up In our next blog post in this series, we will explore the SF Wealth example application and demonstrate some iHub capabilities particularly well-suited to financial institutions. Subscribe (at left) and you’ll be notified when that post and others are published.  

Read More

See Big Data Analytics in Action

Not since the mashup of chocolate and peanut butter have people been so excited about two great products that fit great together: Analytics and the Cloud. Earlier this month, OpenText announced Big Data Analytics in The Cloud,  an all-in-one software appliance built for business analysts that need to access, blend, explore, and analyze data fast without depending on IT or data experts. The need for Big Data Analytics should be obvious. Businesses need to understand their data requirements. They need to digest hundreds of tables and billions of rows of data from disparate data sources. With a powerful analytics tool on their side, companies speed up their time to value with the ability to integrate data from multiple sources to get a single view of their business. No complex data modeling or coding is required. They can clean, enrich and analyze billions of records in seconds and apply advanced and predictive techniques in a visual, intuitive way. But seeing is believing. This is why we have assembled a demonstration video that shows just how Big Data Analytics works and some scenarios that may mirror your own needs. Check out the demonstration here: And if you are interested testing out Big Data Analytics yourself, we have also a free trial available.  

Read More

Data Driven Digest for September 18: Money and Finance

This week marks the 133 anniversary of the opening of the Pacific Stock Exchange in San Francisco. The establishment was created to serve the interest of businesses that struck it rich mining for gold during the California Gold Rush. Nowadays, businesses mine for data hoping to strike it rich by analyzing that data for clues about how to best serve their customers, streamline their operations, or gain a competitive advantage. In honor of those financial pioneers, this week we offer three different visualizations of financial data. Eureka! U.S. Fiscal Responsibility   In 1789, the United States established its first loan to pay salaries of the existing and future presidents and the Congress. As our friend Katy French (@katyifrench) posted in Visual News this week, bean counters in Washington kept great records and even produced stunning visualizations to represent trends. The graphic above represents the Fiscal Chart of Debt and Expenditures by the U.S. Government between 1789 and 1870. Note the spikes in military spending during the War of 1812 and Civil War as well as the first major accumulation of debt in 1861.   Euro Spending How do Europeans spend their paychecks? That was the premise of a recent data plot developed by The Economist (@TheEconomist). Based on data sets from Eurostat entitled Final consumption expenditure of households by consumption purpose, The Economist found life in the Euro zone is quite diverse. Living in Lithuania? Your budget is dominated by food and clothes. Lithuanians also spend more per capita on alcohol and tobacco than the rest of Europe. Meeting in Malta? Forget about eating at home. Nearly 20 percent of Maltese spending goes toward restaurants and hotels. Spaniards spend the least on their transportation. Germans spend more on their furnishings than their E.U. neighbors   World Population Based on Income Our friends over at Pew Research Center (@PewResearch) have come up with an interactive visualization based around the paradigms of income and how it relates to world population. For example, the map above shows the density of people living under what they term as a middle income. By middle income, that means your daily wages are between $10.01 and $20. According to the map, 13 percent of the 7+ billion people in the world are middle income. The map has a second option that reveals the percentage point change in that population between 2000 and 2011. It’s a fascinating study on both financial statistics as well as data maps. The income groups are defined as follows: The poor live on $2 or less daily, low income on $2.01-10, middle-income on $10.01-20, upper-middle income on $20.01-50, and high income on more than $50; figures expressed in 2011 purchasing power parities in 2011 prices.

Read More

Digital Engagement: A New Business Requirement

Digital engagement isn’t an option anymore, it’s a requirement. Today’s consumers are savvy and fickle, and companies must work to earn their loyalty. They’re demanding more from the brands they love, and their tolerance for anything but a seamless, engaging, and compelling experience is flagging. In a digital world, organizations must digitize their customer journeys, from initial interest through to purchase and follow-on service or support. The best way to do this is to shift to a digital marketing strategy. One that creates consistent and compelling customer experiences at every touchpoint through omni-channel delivery, responsive design, and targeted communications and information. Digital technologies have introduced new customer touchpoints and increased opportunities to engage. Since consumers often use more than one channel to interact with a brand (in some instances they use five or six), delivering uniform and relevant messages across all channels is crucial for return on marketing investments and customer satisfaction. Omni-channel focuses on meeting consumer needs by pulling together programs to provide a cohesive brand experience across channels, platforms, and devices. To borrow from Bruce Lee, digital design should “be like water”. You put water into a cup, it becomes the cup. You put water into a bottle, it becomes the bottle. You put water into a teapot, it becomes the teapot. The same holds true for digital experiences. The transition from desktop to device to point-of-sale should be fluid. This is achieved through responsive design. Customers don’t see individual devices or channels; they look for a consistent and familiar brand experience that delivers relevant content. Nirvana on the customer journey is realized when a company anticipates the needs and wants of a customer and serves up targeted and tailored content, products, or services, in the moment of need, wherever the customer is. Organizations that can predict customer behavior have a better chance at fulfilling consumer needs. Analytics—or analyzing data collected across various touchpoints of the customer journey (transactions, interactions, social media sites, and devices) helps organizations discover valuable customer insights so that they can offer more personalized and satisfying experiences. The most effective way to target different audiences is to use messages that focus on products and services with the greatest appeal for each segment. Using dynamically generated customer communications, organizations can create and automate their marketing campaigns. When correspondence is part of a digitized process, end results are gains in efficiency and the ability to create superior customer experiences. As one of the foundational suites for Enterprise Information Management (EIM), Customer Experience Management (CEM) aims to create a richer, more interactive online experience across multiple channels without sacrificing requirements for compliance and information governance. CEM brings together all of the technologies required to re-architect back-office systems, consolidate customer data, and create digitized front-end experiences. Digital engagement starts inside the firewall and extends outside the enterprise and all along the supply chain. In the next post in this series, I’ll explore how the supply chain is being disrupted and how enterprises can digitize key processes for greater collaboration, information exchange, and business agility. Find out how you can capitalize on digital disruption. To learn more, read my book, Digital: Disrupt or Die.

Read More

Exploring iHub Examples – First in a Series

There’s a big-box retail store not far from the OpenText Analytics office in San Mateo, California. Some of us (ahem) have been known to visit there at lunchtime to score free samples for lunch. And why not? Most people like to get something of value for free. If you download the 45-day free trial of OpenText Information Hub (iHub), you know the feeling. Not only do you get to try out a full-featured version of the software for free, but you’re also given a number of free sample applications that preview some of the software’s remarkable capabilities. To help you steer your cart around the wide aisles to find the good stuff, we have prepared a series of blog posts exploring what each sample apps is and what you should look for when you use it. We’ll count on you to imagine how the capability it demonstrates could be used in your organization. One other thing: You and your development team can learn a lot more about how these sample apps are constructed – and how they work – by exploring them in Analytics Designer, the free companion design tool for iHub. Brian Combs has published a step-by-step guide to help you do this. Finding the Samples When you launch iHub the first time you’re greeted with a main screen showing just one item: an HTML file called Examples. Click on it and you’ll see the Sample Content screen below – it’s your jumping-off point for all of the samples. (screenshot) The first sample we’ll explore is labeled Other Applications and can be found in the upper-right corner of the Sample Content screen. Click it and you’ll see three sample visualizations and one dashboard.  These samples (and others) are based on Classic Models or SF Wealth, two of the sample databases that come with iHub. They all present data in a clean, uncluttered format that invites further exploration. Customer Revenue Metrics What it is: A report with a bar chart, a table of top customers (with scorecard arrows showing trends), and pie and bar charts that break out revenue in different ways. What to look for: All of the elements of this report are interactive, so alter them to see what happens. For example, you can change the date range in the bar chart three different ways: by clicking the Zoom setting (upper left), by typing dates in the “from” and “to” boxes, or by moving the slider below the chart. (Modify one of these controls and the other two change in response.) Now click on any bar in the top bar chart for details on a single month. Next, hover over any segment of the pie chart; when a data point pops up, click on it for more detail. Client Investment Portfolio What it is: In essence, this is a periodic statement – like the one you might get from your investment advisor or broker – on steroids. What to look for: This report is an ideal place to explore the power of iHub’s Interactive Viewer. Click the menu button in the upper left corner of the report and select Enable Interactivity. Then click on %Change (the sixth column) and extra controls will appear to filter, sort, and otherwise modify the table. You can use these to sort the entire table based on a single parameter. (When you do this, the right column of the table – with its red and green tags that display the data in scorecard style – will sort accordingly.) Enabling Interactivity unlocks a wide range of capabilities that vary depending on the data or visualization you’re working with. One other thing: click the name of one of the stocks in the portfolio (such as Coca Cola Company), and a new tab will open with the Yahoo Finance page for that asset. This shows how reports in iHub can seamlessly connect with external assets. Top Sales Performers What it is: A ton of data about salespeople, presented in a compact, efficient format. What to look for: While your eye may be drawn to the radar chart at the top of the first page, a sales manager might find the sub-tables under the chart more compelling. These tables demonstrate how complex, multi-layered data can be aggregated and organized in a number of different ways: The salespeople are ranked, and their total sales are calculated. Within that level of organization, each salesperson’s top customers and top products are listed in order. This type of consolidated, interactive information is invaluable to people who manage large, distributed sales forces. Customer Sales Dashboard What it is: A basic interactive dashboard of sales data. What to look for: One big distinction between dashboards and reports is the presence of selectors on dashboards. In this simple example, the selectors are on the left, labeled Sales Territories, Customer Countries, and Year. Click on any element within those selectors and watch the data visualizations (also called Gadgets) on the dashboard respond. Now look at the Historical Revenue Analysis gadget in the lower right corner of the dashboard. If you find it difficult to distinguish between individual data lines in the graph, click the triangle in the gadget’s upper-right corner and choose Maximize. The graph now fills the screen for easier exploration. Next Up In our next blog post in this series, we will walk you through the example called Integration Framework. Geared toward ISVs, this example showcases various capabilities iHub provides for embedding content within an application.   photo courtesy of Sarah Murray

Read More

3 Questions: Content Marketing Expert Robert Rose on the Power of Analytics

Think your organization can tell the difference between good marketing content and great content? Only 36 percent of B2B marketers surveyed in 2014 by the Content Marketing Institute said they were effective at content marketing. To help increase its effectiveness, marketing experts suggest improving content measurement methods. White papers, brochures and blogs get the message out. Analytics illustrates a richer story. Robert Rose is the Chief Strategy Officer for the Content Marketing Institute and a senior contributing consultant for Digital Clarity Group. Robert’s highly anticipated second book – Experiences: The Seventh Era of Marketing is now available. His first book, Managing Content Marketing, spent two weeks as a top ten marketing book on Amazon.com and is generally considered to be the “owner’s manual” of the Content Marketing process. Robert is also the co-host of the podcast PNR’s This Old Marketing, the Number 1 podcast as reviewed by MarketingPodcasts.com We sat down with Robert to discuss the importance of transforming content into digital and the best ways to optimize value from analyzing that content. OpenText: With the world migrating towards a digital-first approach, talk about the importance of content-driven experiences. How should marketing, and other departments, optimize their operations to gain the most value of their digital assets? Robert Rose: The real trend is that content-driven experiences are the differentiation of the entire business these days. Whether you look at this as a layer of product development, an element of marketing – or the new way that you handle customer service, consumers now expect a better experience at any part of their particular journey. This means that marketing – and the development of content-driven experiences – must stretch across the entire customer journey. So, this inherently means that the business has to evolve “content” as a strategic asset.  It can simply no longer be just a byproduct of what people produce as part of their jobs – but must be cohesively created, managed, published, optimized and measured as a function in the business. And, in order to do that – the organization’s first step is to actually look at each of those tasks as a recognized function in the business. It must have actual organization, real responsibility, budget and measurability. OpenText: The intersection of digital content, cloud delivery and Big Data analysis seems like the next step for so many organizations. What recommendations can you give to decision makers in their quest for a digital content supply chain? Robert Rose: The key is to simplify. A great content-as-supply-chain process should actually reduce the amount of content being produced, but optimize its quality and efficacy. This means, ultimately, that the data it produces becomes higher quality and get be used to derive better meaning, and thus greater insight into how to improve the experiences being created.  The classic mistake that most businesses make is they create content in order to facilitate the sales, marketing and service of products – and then simply can’t keep up with the cadence that the product/service requires. Instead, they need to start with the customer, and the experience they’re trying to deliver – then work backwards to see how content can be created to build that experience. OpenText: There are many organizations that are successful in transforming their content and measuring its effectiveness. What are you top favorites and what made them so successful? Robert Rose: I think my current favorite is what Motorola Solutions has done by integrating technology and marketing into one common department. Eduardo Conrado is the Chief Innovation Officer (and wrote the introduction to my newest book). He recognized as the head of marketing and IT that both were truly focused on the same goal; creating a more compelling customer experience. So, he merged both of them together so that they work together. As he says, this really does create an environment where “technology can help you get closer to the customer.” For more insight, Robert’s strategy whitepaper entitled, The Marketing Transformation: From Managing Campaigns to Orchestrating Experiences can be found at OpenText.

Read More

Opening iHub Examples in Analytics Designer

If you’re just getting started with OpenText™ Information Hub (iHub), you’ve probably found the Examples that ship with the product. A collection of sample applications, dashboards and reports, the Examples are meant to inspire you as you create your own iHub-based projects. The best way to understand how these sample applications work is to open them in Analytics Designer, the free commercial-grade companion design tool for iHub. Download Analytics Designer here. This blog post shows you how to get the Sample Application content out of iHub and load it into Analytics Designer. The following steps are based on iHub 3.1.1 Trial Edition and Analytics Designer 4.4. Download iHub, Trial Edition here. 1. Launch the Analytics Designer, and make sure that no report designs, data objects, or other elements are open. 2. Create a New BIRT Project in Analytics Designer, as shown in this screenshot: 3. Select Blank Project as shown below, and give your new project the same name as the application folder in iHub. (For this example we are using the name “Sample Application,” because that is the specific iHub example that we will move – but the process for the other examples is essentially the same.) Then click Finish. 4. Create a server profile for iHub. Select the Server Explorer tab in the Analytics Designer, then click the New Server Profile button (shown with a red arrow below). Make sure that you select the Servers icon (as shown) before you click  the New Server Profile button. 5. Provide a name for your Server Profile and enter the server connection information. For this example I am using the IP address for the iHub 3.1.1 virtual machine that I am running (marked with a red box, below). I am also using the default settings: Administrator for the User name, and no Password. (If you’ve changed the defaults on your iHub instance, you’ll need to use your own credentials.) When you’re done, click  Finish. 6. Make sure that the project you want to download the file into  – the one you created in Step 2 – is selected. 7. From the Analytics Designer’s File menu, select Download… 8. In the download dialog, check Download Files. For the Download Location, click the Browse button and navigate to the project where you want the sample application to reside. If you only have one project, you may only see a “/” for the path, as seen in the screenshot under step 9d below. 9. To download all the files and folders that are in the Sample Application folder in iHub, do the following: a. Check Sample Application. b. Uncheck the first folder under Sample Application. c. Now re-check the first folder under Sample Application. You will see that all the folders and files under Sample Application are now checked, but that the Sample Application folder itself is not (as shown below). By doing this, you make sure that you only download the folders and files in the Sample Application folder into your project. d. Click Download. 9. If you get a warning saying that a file already exists, click Yes to replace the file. 10. You can now expand your project in Analytics Designer and see all the content you just download. Now that you have the iHub Sample Applications in Analytics Designer, you can explore them, modify them, and even borrow from them when you create your own projects. If you’re just learning about software from OpenText Analytics, you can download iHub, Trial Edition here. You can download Analytics Designer here – this companion design tool for iHub is free.

Read More

Data Driven Digest for September 11: Clouds

There’s big news at OpenText this week: Big Data Analytics is now available on the OpenText Cloud. This exciting event got us thinking about actual clouds – you know, the kind up in the sky – and other things that fly through the air. Not surprisingly, there are countless great data visualizations related to clouds and weather, so it was tough to choose just three to share. Swirling Winds: Cameron Beccario (@cambecc) has created a stunning animated visualization, called simply earth, that does a beautiful job of presenting diverse atmospheric data. The visualization blends four data sources – weather conditions, ocean currents, ocean surface temperatures, and ocean waves – each of a different time interval. Click on the word “earth” in the lower left corner of the screen, and controls pop up (as shown in the screenshot above) that let you control the resulting visualization. By the way, the care Beccario takes to document his work is as impressive as the visualization itself. Fly-By: Weather radar systems are (obviously) designed to monitor and record weather. But scientists at the European Network for the Radar Surveillance of Animal Movement (ENRAM) have developed ways to use meteorological technology to monitor bird migration.  Last summer they developed a proof of concept (screenshot above) showing bird movement in the Netherlands over just a few days. The idea has taken off (so to speak), and now biodiversity scientists at the Netherlands Research Institute for Nature and Forest (INBO) are exploring ways to use weather instruments to track other species. Check out the POC, and read more about the work on the INBO blog. Data Flow: For Californians in a drought, no meteorological phenomenon is more important than El Niño. Warming waters in the Pacific Ocean affect the weather worldwide, and often help to bring needed precipitation to the western United States. In an effort to understand this year’s El Niño, Matt Rehme of the National Center for Atmospheric Research (NCAR) released a video comparing the epic 1997 El Niño with the one brewing this year. The result (embedded above, and linked here) demonstrates both the weaknesses and strengths of video for data visualization. One obvious shortcoming is that you can’t explore the data in depth; you just let the image flow by. But ease of consumption and sharing are a video strength; indeed, Rehme’s video has been viewed some 61,000 times in less than a week.  

Read More

Analytics in the Cloud: No Hardware, No Coding, No Punch Cards

Data analysis has come a long way. The British Census of 1911 was the first to use automatic data processing with the advent of punch cards that could be sorted by machine. It was also the first census to ask multiple-choice questions, which helped Britain gather and analyze data on various segments, such as the number of carpenters compared to butchers in the country. Since then, data analysis has become so ubiquitous that more data is generated online every second today than was stored in the entire Internet 20 years ago. If your business is going to survive this digital transformation, it needs to quickly access, blend, explore and analyze all data without depending on IT or data experts. The good news is that OpenText is addressing those needs with the launch of its Big Data Analytics in the Cloud. To address the needs of companies seeking Advanced Analytics (a $1.4 billion market, according to estimates from IDC), Big Data Analytics has a built-in high-speed columnar database that provides thousands of times faster performance than traditional relational databases. The software incorporates statistical algorithms, making it easy to do profiling, mapping, clustering, forecasting, decision trees and more without programming. Delivering these capabilities as a managed cloud service reduces investment in infrastructure and maintenance staff. In a nutshell: no hardware to buy and no coding required. You can get all of your data in a single view and the ability to analyze billions of records in seconds. Powerful Enough For All Your Data Big Data Analytics is engineered to read virtually any data source. It includes native connectors for popular SQL databases, an Open Database Connectivity (ODBC) driver for creating custom connections, built-in ability to access flat files and CSV files, and a remote data provider option for loading files using a web address. On top of these powerful tools, Big Data Analytics gives everyday users access to advanced analytic algorithms formerly available only to data scientists. These tools and algorithms are optimized and hard-wired into the product and accessed via a toolbar in the Analysis window (as seen below).   Crosstab allows you to cross multiple data fields – either within the same database table or from different tables – and display the results as dynamic tables and graphics. Venn diagrams visually identify coincidences and differences between up to five data segments for rapid discovery. Bubble diagrams show the distribution of categorical data across two axes of numeric variables. A third variable can affect the size of the bubbles that represent the data. Results of bubble analyses can also be viewed in table form. Evolution analysis shows data progression over time. Visually, evolution analyses resemble bubble diagrams, but the spheres representing data move to show time passing. The user can freeze playback and adjust the time interval. Profile analysis groups values and determines relatedness to a profile segment. Users can easily see how individual attributes contribute to the overall profile. Results are presented in a table that visually represents statistical relationships (known as Z-score). Map analysis displays data on a choropleth map, in which different colors or shades represent the magnitude of data values. Multiple maps with region names are encoded in the product, and new maps can be added. Pareto analysis is the algorithmic expression of the 80/20 rule. It enables users to see if and how their data conforms to that rule.  If only Britain had this kind of technology back in 1911. Perhaps, they could have predicted the British Music Invasion of the 60s or that one day, Tim Berners-Lee would define the World Wide Web. To find out more about how to optimize your data for a digital future, I recommend attending one of the upcoming Big Data Analytics webinars on September 22 or October 15.

Read More

Data Driven Digest for September 4: The Seasons

It’s Labor Day Weekend in North America – the traditional end of summer, if not the season’s end as marked by the equinox. To acknowledge the impending shift to autumn, this week’s Data Driven Digest is about seasons – how they affect vegetation, energy use, and air quality. Fall Back: The very first Data Driven Digest we published almost a year ago contained a link to the lovely Fall Foliage Map published by SmokyMountains.com. So I was delighted to hear from David Angotti (@DavidAngotti) that the 2015 Fall Foliage Map is now online. Based on more than 37,000 data points, the map visualizes fall color creeping across the United States; move a slider (representing time) and watch the colors change. “The data behind the map is primarily a conglomeration of NOAA precipitation forecasts, daylight and temperature forecasts, historical precipitation data for the current year, and a deduction of many government and private party data sources available online,” Melton says developer Wes Melton. “Once we have accumulated the data, there are manual changes we then make to the dataset based on our knowledge of the topic.”   Power Up: Energy use shifts logically with the seasons: In summer it’s used for cooling, while in winter energy is applied to lighting and heating. What’s less apparent is where the energy comes from. The Washington Post team of John Muyskens, Dan Keating and Samuel Granados published a terrific interactive site that explores how the United States generates electricity. Each circle on the map represents a power plant; the size of the circle represents the plant’s output, and the color of the circle reveals its power source. Roll over the main map (click through for the interactive version) to see a breakdown by state. Farther down in the site you’ll find an interactive bar chart showing the same data in a very different manner. If you’re interested in exploring more energy-related data, the U.S. Energy Information Administration’s Short-Term Energy Outlook publishes incredibly detailed reports on a monthly basis.   Smoke Signals: Our colleague Michelle Ballou, who lives smack dab in the middle of Washington State, visited our San Mateo office this week and talked about the choking smoke from seasonal wildfires in her areas. Coincidentally, my colleague Michael Singer (@MichaelSinger) learned about BlueSky . A project of the United States Forest Service Research & Development arm, BlueSky links a variety of independent data models – fire information, fuel loading, fire consumption, fire emissions, and smoke dispersion – and uses predictive models to simulate and animate the cumulative impacts of smoke on air quality from forest, agricultural, and range fires. The Forest Service, by the way, collects and shares a wealth of public data sets. Seeing Spots (Bonus Item): Shout out to Massimiliano Mauro (@MM_cco), an app designer for Wired Italia, for the stylish and functional 3D data visualization shown above. Called 3D and D3.js, it is a Braille-inspired three-dimensional data visualization that shows invisible air pollution (unlike the smoke in Michelle’s town, which is very visible). Each dot represents a day, and days are arranged by season to better reveal patterns in the data.

Read More

How IoT Based Analytics Will Drive Future Supply Chain Operations

Over the past couple of years we have seen an exponential growth in interest around the Internet of Things (IoT). My interest in this space started at Cisco’s IoT World Forum in Barcelona in late 2013.  Back then many of the software and solution vendors were just starting to define their IoT strategies due to the various estimates that analysts had put out about the expected value of the IoT market over the next decade. There were two interesting IoT related announcements this week, firstly GE placing all their IT and software solutions into a new division called GE Digital. Slight irony here in that this is the second time GE has done this, the first time was when they established and then spun off their former IT division which later became GXS!  The second announcement came yesterday at Salesforce’s annual conference where they announced their own cloud based IoT platform.  So the IoT cloud market is certainly hotting up. In 2013 I posted my first blog discussing where I believed IoT would impact supply chain operations and from what I could tell back then, based on the number of IoT and Supply Chain articles that had been published, I was early to predict how IoT would transform tomorrow’s supply chains. Many argue that some components of an IoT environment, such as RFID tags, have been around for many years and in fact IoT has now given RFID tags a stronger sense of purpose.  However other technologies such as Big Data Analytics are really only just starting to be applied in the supply chain space. For me, I see three areas where IoT will add value to supply chain operations, I call these the ‘Three Ps’ of supply chain focused IoT, namely Pervasive Visibility, Proactive Replenishment and Predictive  Maintenance. One common aspect to all three of the above scenarios is big data analytics.  Earlier this year OpenText acquired a leading provider of embedded analytics solutions, Actuate.  Over the past few months we have been busy embracing the world of big data analytics and recently announced a cloud based analytics offering. This is quite a game changer in the big data analytics market as companies look to take their first steps into the world of analytics and OpenText Big Data Analytics in the cloud allows companies to scale their analytics platform over time and align with the size of the analytics project being undertaken. In fact yesterday, OpenText was ranked number three in a new report from Dresner Advisory Services, they looked at the Business Intelligence market in the context of IoT. It is worth noting that the chart and vendor analysis conducted by Dresner was carried out before the launch of our cloud based analytics solution, so we would probably have been ranked higher than number three out of seventeen vendors.  When you consider the size of the analytics market and the number of vendors in the space, this is quite an achievement for our solution and it puts us in a good position for companies looking to process the huge volumes of data coming off millions of connected devices in the future. OpenText Big Data Analytics is a core component of OpenText’s cloud strategy and early last year OpenText acquired another key cloud solution provider GXS.  OpenText now operates the world’s largest B2B integration network with over 600,000 companies connected to the network and these companies are processing over 16billion transactions per year.  Now wait a minute, 16billion transactions!, now that is a lot of information flowing across our network that could add a lot of value to companies if they had a way of analysing the transactions in real time. As you would imagine we are busy looking at how our Trading Grid platform could leverage the capabilities of our new cloud based analytics solution. I have spent the past two years keeping a close eye on the IoT market and it is great to think that our cloud based analytics solution provides a stepping stone into the ever growing IoT market.  But what happens when you bring the world of IoT and supply chains together?  I wanted to use the following diagram to explain how OpenText Analytics and Trading Grid could in the near future provide support for the three supply chain scenarios that I mentioned earlier, namely pervasive visibility, proactive replenishment and predictive maintenance. The diagram below illustrates a desktop demonstration of how consumption trends from a connected device can help to initiate a ‘purchase to pay’ process.  When I say purchase to pay I am talking about an order being created, goods being delivered and then payment made to the supplier.  Let me now break this diagram down into a few key steps. The first stage is the connected device itself, now it could be any type of connected device, but for this example I have chosen a WiFi enabled coffee machine. In addition, for the purposes of this demonstration, a connected coffee capsule dispenser, so as you remove a capsule this will be recognized by a proximity sensor placed underneath the capsule. The second stage is to then capture the consumption trends from the coffee machine.  So as each capsule is taken from the dispenser, a signal would be sent to OpenText Analytics which will essentially be used in this case to monitor consumption patterns and overtime trend related information and graphs etc can be displayed. The key step in this process is when OpenText Analytics detects that a certain number of capsules have been used and an order can be placed via Trading Grid for replacement capsules to be delivered from an outside supplier. This in essence is Proactive Replenishment, where analytics data is driving the ordering process. Back in January this year an article on Forbes.com discussed how in the future connected devices would potentially be able to initiate their own procurement process.  Thus taking manual ordering of replacement goods out of the supply chain process.  Now we are some way off achieving this at the moment but the IoT industry is heading in this direction. For now though a trigger from OpenText Analytics would alert a user to create a Purchase Order for ordering replacement coffee capsules. This ordering process would be initiated through one of our SaaS applications on Trading Grid and this application, Active Orders would also monitor the end to end life cycle of the order.  Mobile access to the progress of the order from the supplier to point of delivery would be available via a mobile app. The order for the capsules is received by the supplier, represented below by a robot arm, which selects the replacement capsules from a rotary capsule dispenser and then loads them on transport provided by the 3PL carrier. Now over time sensors on the robot arm would detect any potential failures with its operation.  From a maintenance point of view, the operational information coming from the sensors on the robot arm would be fed into our analytics platform and overtime you would be able to predict when a part of the robot is likely to fail.  In the real world you would then initiate a repair before the robot fails and hence your supply chain operations are not interrupted in anyway.  This is a perfect example, albeit scaled down of how IoT can drive Predictive Maintenance procedures.  In fact predictive maintenance is widely regarded as one of the most important industrial applications for IoT at this moment in time. For the purposes of this example the 3PL carrier is operating a model train!, which will carry the capsules to coffee machine on the other side of the table.  The location of the train would be monitored via an RFID tag attached to the train. The potential for improving end to end supply chain visibility using IoT and connected 3PL providers is huge and Cisco and DHL recently released a white paper discussing this opportunity. The RFID tags in this case are being used for the purposes of this demonstration but in real life a combination of RFID tags and GPS devices would be used to track the shipments. The ability to connect every piece of supply chain equipment, whether fork lift truck, lorry and pallets etc will transform supply chain visibility and will contribute towards the Pervasive Visibility across an end to end supply chain. So there you have it, a very simple example of how IoT could impact future supply chains.  The IoT market is moving incredibly quickly and who knows what new technology will be introduced over the coming years, but one thing is for sure OpenText can now provide two key components of the IoT enabled supply chain, OpenText Big Data Analytics and OpenText Trading Grid.  The world of B2B integration just got exciting.

Read More

Exciting Times for Payments Professionals!

There can be no debating the fact that the past six months has seen more change in the payments world than any six month period in history. So many new developments underway or proposed; so many exciting possibilities for a faster, safer, more efficient, more competitive and more customer friendly payments environment. Pick a region, pick a form of payment, pick a back-office or settlement function, pick a channel – it’s all changing. The vast scope of changes in payments is going to impact banks, regulators, payment system operators, technology service providers, merchants and businesses of all sizes and consumers. It is going to fundamentally change relationships and business practices that have remained somewhat static, despite technological advances, for centuries (yes, centuries!). The thought of how any one entity incorporates these changes into its business to maximize the benefits while minimizing disruption is overwhelming. So, as we enter this environment in which everything seems to be changing in overlapping timeframes, what is a payments professional to do? How does anyone who is concerned with making or receiving payments (isn’t that all of us?) make sense of this? No doubt, one could spend all day reading payments newsletters, watching webinars, reading blogs like this one, following experts on Twitter. Unfortunately, that’s just not practical for anyone other than a full-time payments geek. For the rest of us, it means relying on partners whose business it is to ensure that your particular organization is prepared for the changes that are most relevant. That might be a bank, a trade association, a technology provider, a regulator, an industry analyst or someone else. In fact, for most of you, it probably involves several of those. No matter what your role in payments, the changes will be vast but they boil down to a few themes: Electronic payments will be processed, cleared and settled more quickly New channels will be introduced providing consumers and businesses with easier to use and more convenient methods for initiating payments and processing payment information Security methods will increasingly rely on biometrics and less on passwords Global standards (specifically ISO20022) will be adopted in virtually all payment systems in the developed world and in many emerging markets facilitating inter-operability for cross-border payments Competition from non-traditional players will become the norm in (almost) every aspect of payments resulting in massive regulatory changes to ensure the safety and soundness of payment systems An increasing focus on payment-related data to provide better information and context to all participants for individual transactions as well as for trend analysis. Successful navigation of the changes that are coming requires a comprehensive strategy to future-proof your payments environment. Where to start? It’s always best to take a fresh look at your existing environment, understanding how you currently use the existing payment systems and identifying areas for improvement. Then, start to engage your partners who are closely following and/or are involved in the most relevant efforts to change payments. Attend industry events such as SIBOS 2015 in Singapore or the 2015 AFP Annual Conference in Denver. Engage with NACHA’s Payments Innovation Alliance or the EBA or whichever national or regional organization that is devoting time to payments modernization where you operate. It is an exciting time to be a payments professional. That just might be the understatement of the year!

Read More

Data Driven Digest for August 28: Treemaps

The first treemap was created around 1990, for a reason that seems laughable today: 14 people in a University of Maryland computer lab shared an 80-megabyte disk drive, and one of them – professor Ben Shneiderman – wanted to know which individuals, and which files, took up the most space. After considering circular, triangular, and rectangular representations, Prof. Shneiderman came up with the nested, colored rectangle format we use today. (His history of treemaps is fun reading if you  want to learn more.) Of course, treemaps have proven valuable for much more than determining  who’s hogging the hard drive, as evidenced by the examples below. Leaves of Green: The venerable line chart was the most common representation of Monday morning’s precipitous drop in the U.S. stock market, but we gravitated toward the treemap above, published by finviz. This visualization represents the stocks in the S&P 500, categorized by sector and sized by market cap; color represents performance. (This screenshot was taken at 9:45 a.m. PDT on Monday; look at all that red.) You can hover over any individual stock to get details, including a comparison against other stocks in its sector; simply double click to get a detailed report. You can also change the time horizon and other metrics the chart displays.  It’s endlessly interactive and interesting to use. Repping Fragmentation: Though app developer OpenSignal is best known for its wireless coverage maps, the company also does a great job of collecting non-geographic data from its apps.  The treemap above is from an OpenSignal report about device fragmentation in the Android market published earlier this month. The treemap catalogs the 24,093 different types of Android devices that downloaded the OpenSignal app over just a few months in 2015. (No wonder your mobile app-developer friends look tired all the time.) The various colors represent device brands; hover over any square to see the make and model of the device. The segments of the treemap sort from large to small – upper left to lower right – but another visualization later in the report presents the same data sorted by brand. You can also see how much the market has changed from August 2014 to August 2015 with the click of a button. All Wet: The Food and Agriculture Organization of the United Nations collects copious data in its mission to eliminate hunger, fight poverty, and support sustainability. A recent report on global irrigation uses treemaps very effectively to visualize Big Data about water use in agriculture: what irrigation technologies are in use,  what  sources of water are tapped, and how much geographic area is under irrigation worldwide. (The unit for determining rectangle size in the treemap is hectares, not water volume.) A click anywhere on a treemap brings up the data underlying the chart, including links to download the data for your own analysis. Getting cultured (bonus item): Pantheon, a website created by MIT Media Lab in 2014, shows how a treemap can support the study of history. Pantheon visualizes “historical cultural production” – you choose a country, a date range, and other parameters, and the site creates a treemap showing well-known people from that country, grouped by domain. (Domains include historian, religious figure, and pirate. Yes, pirate.)  Or you can flip this around: start with the domain, choose a timeframe, and discover which countries that have produced the most prominent people. In all cases, more details are a click away. The concept of Pantheon is easier to understand than to explain, so if you don’t get it from this write-up, click through and play with it. By the way, the Custom Visualizations capability in OpenText Information Hub (iHub) enables you to create treemaps from your own data in iHub using either D3.js or Highcharts. Check out the post about using Custom Visualizations.  

Read More

Here’s Your No. 1 Tool for Fraud Detection

In the song Talk About the Blues, the experimental band The Jon Spencer Blues Explosion declares that “the blues is number 1.”  If you’re a blues fan, you probably associate the number 1 with Jon Spencer. But if you’re interested in fraudulent or anomalous numbers, you might just associate it with Frank Benford. Here’s why: Back in 1938, Benford observed that 1 is more likely than any other number to be the first digit (also called the most significant digit) of a natural number. He determined this based on analysis of 20 different sources of numbers, ranging from articles in Reader’s Digest to population sizes to drainage rates of rivers. At a remarkable rate, the first digit of the numbers Benford studied was either a 1 or a 2 – and most frequently a 1. But Benford wasn’t the first person to discover this. The astronomer Simon Newcomb noticed it in 1881, while thumbing through books of logarithm tables. Newcomb noticed that pages with log tables for numbers beginning with 1 or 2 were grubbier than the pages for numbers beginning with 8 or 9. After some mathematical exploration, Newcomb proposed a law stating that natural numbers were much more likely to begin with a 1 or a 2 than with any other digits. In fact, he said that natural numbers had 1 as their first digit about 30 percent of the time. Newcomb’s observation wasn’t discussed much for more than 50 years. Then Benford (who worked  as a physicist for the General Electric Company), tested Newcomb’s law on 20 different data sets. Based on his calculations (his distribution is shown above), Benford declared that Newcomb’s law was “certain” – and, without hesitation, he applied his own name to the phenomenon. (Smart guy!) Now known as Benford’s Law, the idea has come to acquire an aura of mystery. After all, if a collection of numbers is truly natural – that is, “occurring commonly and obviously in nature” – shouldn’t their first digits be identically distributed across all numbers from 1 to 9? Benford’s Law is mysterious, yes, but it works.  It’s now widely used by investigators looking for fraudulent numbers in tax returns, credit card transactions, and other big data sets that are not aggregated. Obviously, it doesn’t work with dates, postal codes and other “preformed” numbers. A helpful basic discussion of Benford’s Law is available via Khan Academy. Its simplicity makes Benford’s Law really easy to apply to automatic audit procedures. You only need to compare the first digit of the set of numbers that you want to analyze against the distribution in Benford’s Law. If certain values in your data deviate from what Benford’s Law dictates, those numbers probably aren’t natural. Instead, they may have been invented or manipulated, and a deeper analysis is required to find the problem. For example, consider the distribution shown in the dots and black line on the chart below. Compare them to the blue bars and numbers, which represent Benford’s distribution. You can clearly see (because we’ve circled it in red) that more than 20 percent of the numbers in the data set have 2 as their significant digit, even though Benford’s Law says they should represent less than 18 percent. This tells us something is fishy, and it may be worthwhile to dig deeper into the underlying numbers. This kind of analysis doesn’t have to end with the most significant digit. You can also analyze the second, third and fourth digits; each has its own distribution that will allow you to isolate possible fraudulent numbers from millions of legitimate transactions for further analysis. You can apply Benford’s Law to your data really easily with OpenText Actuate Big Data Analytics. Just follow this step-by-step guide in our Free Trial Resource Center.  Cross this information with incomes, tax returns, revenues, and financial transactions. If there is something strange or fraudulent in your data, you will find it. And instead of singing the blues like the lip-syncing actors below, you’ll sing the praises of Frank Benford. Number One image by Timothy Krause, via Flickr. 

Read More

Data Driven Digest for August 21

The second* data visualization we all learned in school was the Cartesian coordinate system.  By plotting figures on a two-dimensional graph, we learned the relationship between numbers and space, unlocked patterns in those numbers, and established foundations for understanding algebra and geometry. The simple beauty of X-Y coordinates belies the power they hold; indeed, many of the best data visualizations created today rely on, and build upon, on the Cartesian plane concept to show complex data sets. Here are three examples. (Note that none of these are textbook Cartesian visualizations, because the X and Y axes represent different units.) Back to School: Our favorite “Data Tinkerer,” Randy Olson, published a blog post this week exploring correlations between earnings, gender, and college major. Using data from the American Community Survey (and building on a FiveThirtyEight article by Ben Casselman), Olson created the graph above to show his findings. Then he generated a variety of graphs (one of which is below) that fit a linear regression onto the data and add bar charts along the graphs’ sides to show quantity along both axes. The results very effectively illuminate more aspects of the same data in a very efficient format. Statistically Significant: Scientists are sometimes accused of adjusting their experiments to yield the answers they want. This practice is called p-hacking (for p-value) and is explained in a fine FiveThirtyEight article by Christie Aschwanden, Science Isn’t Broken – It’s just a hell of a lot harder than we give it credit for. The article is accompanied by the endlessly fun interactive shown above; click through to play with it. As you add or subtract parameters, the data on the Cartesian plane and the linear regression of that data change before your eyes. If you can find a connection that yields a p-value of 0.05 or less, Aschwanden says, you have data that’s suitable for publishing in an academic journal. Click here for a great explanation of p-values. Business Time: At the Harvard Business Review, Ronald Klingebiel and John Joseph delved into whether it’s better to be a pioneer or a follower by studying a very specific slice of data: German mobile-handset makers in the years 2004-2008. Their chart (above) plots many manufacturers along two axes; the number of features on the x axis, and the month of entry into the market along the Y axis. Klingebiel and Joseph then highlight two companies that succeeded (Samsung and Sagem) and two that didn’t (HP and Motorola). The authors’ hypothesis was that a handset manufacturer was more likely to succeed if it came to market early with lots of features, or if it arrived later with fewer, better-focused features. The chart, while very good, would benefit from interactivity; I’d like to hover on any dot to get the company name, and click any dot to get details of how that company performed. Without this context, I must rely on the authors’ definition of success. * The number line is the first data visualization I recall using. 

Read More

Data Driven Digest for August 14

Forgive us in California for being obsessed with water. Our unprecedented drought has caused our brains to focus on the wet stuff, so that’s the theme of this week’s Data Driven Digest. In three data visualizations, we dive into what you would see looking west or east across the ocean; the contours and makeup of the seabed; and the width of rivers throughout North America. Grab a cool glass of water and take a look. Look out: Next time you visit an ocean beach, take a few moments to ponder what’s due east or west from you. Then check out What’s across the ocean from you when you’re at the beach, in 7 fascinating maps, created by Weiyi (Dawn) Cai and Ana Swanson of the Washington Post. The beautiful maps are full of surprises; for example, I was startled to see that Japan is as long (from north to south), as the entire United States, and that Boston and Spain share the same latitude. Thanks to my colleague Michael Singer (@MichaelSinger) for suggesting this item.   Under the Sea: Want a glimpse at what lies beneath the ocean’s surface? Check out the first digital map of the earth’s seafloor, created by scientists in Australia. The map, which available in an engrossing interactive version, shows the contours of the seafloor and the sediments that cover it – ranging from calcareous ooze (light blue in the screenshot above) to sand (bright yellow). Some 14,500 data points, collected over 50 years, were compiled by researchers at University of Sydney in Australia; big data experts at National ICT Australia (NICTA) used the support vector machine model for digitization and mapping.  H/T: I learned of this map via Mia De Graff of the Daily Mail.   Moving upstream: If your travels take you to rivers rather than the ocean, we have a data visualization for you, too: Hydrologists George Allen and Tamlin Pavelsky of the University of North Carolina have spent years compiling a database of North American river widths. Their painstaking task started with countless images from Landsat satellites; they combed through them to find more than 1,750 without ice or cloud cover; then, controlling for seasonal changes, they used Rivwidth, a software program created by Pavelsky and Laurence Smith of UCLA, to calculate the width of every river. Joshua Stevens (@jscarto) of the NASA Earth Observatory then turned their data into a 2050 x 1355-pixel map; click the cropped version above to see it, and read more (including why they did it) at NASA’s Earth Observatory blog.  

Read More

Data Driven Digest for August 7

We love Twitter. We love to tweet, we love to follow data visualization experts, and we’d love to have you follow us at @OT_Analytics. But for all of Twitter’s fun and value, it often makes headlines for the wrong reasons: its users don’t understand it, it’s a takeover target, its corner office has an ejector seat. How Twitter (the company) will address those challenges, nobody knows. But we do know this: Twitter (the service) is a tremendous data source, ripe for analysis and visualization, thanks to some  very cool APIs. Here are three sites that make interesting use of Twitter data.  Do you have a different favorite? Suggest it in the comments. Feeling It: Wondering how the Twitterverse feels right now? Go to the sentiment viz created by Christopher Healey (@chris_g_healey) of North Carolina State University and type in a word or hashtag. (For the screenshot above, we searched for #dataviz.) You’ll see recent tweets using your term, plotted on an oval graph designed to visualize emotional affect. The secret sauce is a sentiment dictionary that Healey uses to analyze the tweets. The application also groups tweets by topics, groups them in a heatmap, creates tag clouds, organizes tweets by time and geography, and shows affinity between tweets, people, hashtags, and URLs.  Be sure to read Healey’s detailed write-up of the project. Chit Chat: Data scientists at Booz Allen hosted a Twitter chat in conjunction with Data Innovation Day earlier this year. For several hours, more than 170 people talked about data science in 140-character chunks, using the hashtag #datascichat. Because it was a conversation among data scientists, it’s almost inevitable that one of the participants, Marc Smith of Connected Action (@marc_smith), went on to visualize the conversation (above).  Click through to see Smith’s graph full-size, read a description of the process, and see the source data. There’s even an interactive version to explore. See Through: Twitter takes pride in its transparency. It receives many requests from governments and legal bodies for information about its users, as well as requests to remove information and enforce copyrights. (Not all requests are honored.) All of these requests are cataloged and visualized in a transparency report, compiled semiannually. In the interactive report, you first choose the type of request (information requests, removal requests, and copyright notices). Then you hover over a map to see how many requests came from a given country and how many of those requests Twitter chose to comply with. Thanks to Katharina Streater (who’s infrequently on Twitter at @KatStreater) for submitting this example.  

Read More

Moving Content to iHub, Trial Edition

Remember the last time you moved from one home to another? Moving can be a pain; it’s time-consuming and stressful, so you probably blocked it from your memory. You definitely didn’t enjoy the process. Fortunately, it’s quick and easy to move content to the just-released OpenText Actuate Information Hub (iHub), Trial Edition from an old instance of iHub, Free Edition (formerly known as BIRT iHub F-Type). You simply download content from one application and upload it to the other. (Kind of like packing and unpacking your possessions, but without all the hassle and bubble wrap.) This post walks you through the process. Download Content and Resources from iHub, Free Edition Downloading content from iHub, Free Edition is easy using iHub’s Information Console. Navigate to the Information Console and select the files and/or folders that you want. By default they’re placed in the Applications folder, but you may have stored them elsewhere. Once you’ve found and selected your content, select Download from the Action drop-down menu, as shown below. If you’re downloading folders of content, or multiple files, iHub will create a zip file containing everything you’ve selected. If you download a single file only, your content will not be in a zip file. In either case, the content will be downloaded into your browser’s download location – typically the download folder. (Make note of its location.) If your content requires shared resources – like Data Objects, data stored in flat files, or libraries – you’ll need to download those things, too. The Resources folder is the default location for these resources. Select the needed items and download them using the same process you used for downloading content. Upload into iHub, Trial Edition Now launch iHub, Trial Edition and navigate to the Information Console. Then click the Upload button (marked with a red arrow in the screenshot below). You will then be taken to the upload file selection screen. Navigate to the content you previously downloaded, and select it. You’ll need to upload content and resources separately. If you’re taking content from a zip file, make sure you select the box next to Expand archive (ZIP or TAR format file) as shown below. Keeping Folders Organized A zip file created by iHub (or any zip file, typically) includes a folder path inside the zip. For example, if you download a folder named CallCenterApp that is within iHub’s Applications folder, the zip will contain an Applications folder that has a CallCenterApp folder in it. When you upload the zip file to the Applications folder in iHub, you will end up with an Applications folder within iHub’s Applications folder (as shown below), and the CallCenterApp folder will be in it. This may cause some problems with when you run the content in CallCenterApp. You can easily fix this problem by moving the CallCenterApp folder up a level, to iHub’s Application folder. Just select CallCenterApp, then choose Move To from the Action drop-down menu, as shown below. You will then get the dialog box shown below. Use it to move the folder and all of its contents to a location immediately under the Applications folder in iHub. When you’re done, you can then delete the extra Applications folder. (This is also done using the Actions drop-down menu.) Another Alternative: Use Analytics Designer You can also use the free Analytics Designer to move content between iHub, Free Edition and iHub, Trial Edition. Download the Analytics Designer here, install it, then load your project into it. You can read more about loading content directly into Analytics Designer in the Downloading files section of the Analytics Designer documentation. Once your project is in Analytics Designer, you can publish it directly into iHub. For more information on how to do this, please see the Deploying applications section of the iHub documentation. If you run into any issues while moving your content, please leave a comment here or post your question in the Developer Forums. Good luck with the move, and thank you for using iHub, Trial Edition! Image credit: Flickr user TheMuuj.

Read More