Building Smart Internet

Less than 5% of the information we browse daily on the internet may be worth our attention. Two reasons contribute to our labeling information as garbage (In most cases it is not).

Reason one is that business sites are not responsive – they do not help the user to find the information needed. They just pour on her/him all the information available.

Reason two is time. Information is neither updated nor removed from websites because companies don't have clear policies for doing this. Companies do not monitor the relevancy level of their web pages because the internet is not yet the battlefield for them.

How serious is this problem? In the corporate world, senior engineers and experts spend at least 25% of their time surfing the internet for information. The result is not guaranteed to be the best. I grappled with this problem for years.

How to extend and increase the speed and the efficiency of information search? Move to the smart internet (SI) enhanced with AI-driven technologies.

These technologies create a layer between the user and the internet that interprets and moderates the user requests and the internet responses. While the internet is stateless, this layer has memory and is therefore stateful.

Statefulness is the foundation for AI application. For example, SI may store the user history, which may be used to predict her/his requests and pre-fetch the data. As the average user makes over 500 clicks per day, the history meaningful for AI technologies may be created just in a week. The above technology is a workhorse for such sites as LinkedIn, Twitter, YouTube, Facebook, etc.

Naturally, SI knows well which business it serves, and any business grows on the newsfeed.

Newsfeed

Newsfeed creates the annotations list of newly published content on the internet, sorted by the topic. It saves the user the work of visiting multiple websites and screening out the relevant information. This work is repetitive boring and time-consuming.

The newsfeed developed by crenger.com is centered on desalination, new projects and tenders, and operation and maintenance. At the newsfeed's core are the text categorization algorithms modified to increase their accuracy.

Currently, crenger.com streams the newsfeed output to the website and the PlantDesigner project execution environment (PEX). Additionally, newsfeed may automatically share the data on social networks like LinkedIn and Twitter, or wrap the data into the newsletter bound for subscribers.

Contextual information search

Any web browser is assumed to know WHAT we search for, none – WHY we do this. WHY describes the problem and its context unknown to the browser. Without context, the search rarely hits the target.

The contextual search problem is known for years. To alleviate it, the search engines try to infer the context from the web pages visited by the user. It is a vicious cycle: the poor search results in imprecise context definition, which, in turn, leads to even poorer search.??

That is the reason why advanced search engines (like Meta) request explicit context from the user to improve search precision. Unfortunately, these engines have a steep learning curve affecting their usability.

Has crenger.com succeeded in solving the above problems??If a search is made from PEX mentioned previously, these problems do not exist as PEX is a context. ??

Annotated information search

Modern search engines are heavily biased on popularity and advertising. I normally skip the first three entries in the search results offered by Google.?Engineers need different types of rating – novelty, and quality. This is a domain of experts. ?They search the internet, select web pages, and annotate them – classify their content and rank their novelty and quality. Annotation or bookmark replaces the real webpage content in searches conducted by other engineers served by experts. (The " Know-How Bitcoins" article explains why professional bookmarks are by far more important than the unguided search of the internet.)

The Crenmarks search engine of crenger.com implements the above approach. To annotate webpages, images, or text excerpts, it includes the Chrome extension. It has a clean, minimalist interface without a learning curve, and is available for free download.??

API-based information search

In the last 10 years, nothing has been accelerating the world economy more than Application Programming Interface (API). It is a door to a huge information repository hidden from web search engines. Unlike webpages intended for human beings, the API-based information request and retrieval are directly executed by a computer using the JSON or REST protocols. Crenger.com uses API extensively for project price adjustment clauses auto-generation. Below are some examples of APIs for information retrieval.?

Twitter API provides broad access to public Twitter data that users – private persons, organizations, and companies - have chosen to share with the world. To search through data, the API offers a high-quality query language simple to learn and use.?This is probably the main reason why Twitter API has so high popularity.

US Bureau of Labor Statistics API?gives the public access to economic data from all BLS programs. Data like Producer Price Indexes and Wages are extensively used in megaproject price indexation.

Eurostat API provides high-quality statistics and data on Europe: industry, trade, and services, environment and energy, economy, and finance. API allows customizing requests for data like filtering on dimensions to retrieve specific data subsets.

Metals-API is a simple and lightweight API for current and historical precious metals rates and over 170 world currencies. The API comes with multiple endpoints, each serving a different use case. Endpoint functionalities include getting the latest exchange rate data for all or a specific set of currencies, converting amounts from one currency to another, retrieving time-series data for one or multiple currencies, and querying the API for daily fluctuation data.

Nasdaq API brings together millions of financial and economic indicators from over 250 sources on one easy-to-use platform. They include metal, energy, oil, gas prices, stock prices, futures, options, financial data, economics, etc.

CommoPrices API provides reliable, up-to-date, and independent data, covering the widest range of commodities and raw materials in the world.

Reprinted from crenger.com