Social Intelligence: Fascinating progress in a new approach to understanding public issues.

In research on governmental and management use of computing in the 1970s, I found that some of the most consequential management information was gained by mining operational data (Dutton and Kraemer 1978; Kraemer, Dutton, and Northrop 1981). For example, there was a complaint process in a large city in the midwest of the United States that aggregated complaints to the city in a single integrated list. Simply by looking at what residents were complaining about, the city managers could get early warnings of problems, such as what departments, functions, and areas of the city were experiencing what problems. It was valuable management information. 

Last week I had a delightful lunch with an old colleague from Macau, Angus Cheong. We were both involved with the World Internet Project (WIP) over the last twenty years, but Angus branched off to also become a more independent researcher and entrepreneur, providing social research support to the government and to business enterprises, often focused on analysing complaints and comments. From this work, he has developed a fascinating multi-national business in Macau and Hong Kong with Chinese and English versions of what I found to be among his most intriguing work on mining social media data.

The tools and processes: All credit to uMax Data Technology Limited, uMax Data. Their tools include: xMiner, a platform for big data analysis; LawMiner, a plaform for legal market insights; HK Plulse, a platform for social listening & analytics. But see: uMaxData at https://www.umaxdata.com/ and DiVoMiner at https://www.divominer.com/en/

As you know, or can imagine, comments of individuals on social media, such as about a political candidate or public issue are scattered across multiple social media posts and platforms, such as on Facebook, TikTok, Instagram, Twitter, and many more sites. The information buried in the comments is highly fragmented and seldom analysed. No longer.

His team has the tools and processes in place to text-mine multiple social media platforms for comments that can be tied to topics, like a candidate or issue, and then content analysed to discern what people in the aggregate are saying in their comments. They call this “intelligent cross-platform data technology”. As he is a social scientist, Angus and his colleagues understand research methods, such as data sampling, testing the reliability of indicators, coding qualitative content, doing automated content and text analysis, as well as statistical analysis, and visualisation. It can be done quite systematically.

Suddenly, information lost in cyberspace is accessible and interpretable in ways that can be informative for marketing, campaign evaluations and management, evaluation of public programmes, customer experiences with products, monitoring brands, sensing responses to public policies, and so forth. It is not totally AI, as their process in using AI and text-mining but to augment human coders. But as coders make decisions, such as this term is equivalent to another term, the analysis can quickly work through the coding decisions to reanalyse and visualise the results.

As it is anonymised and aggregated, so it is not seriously worrisome from a surveillance standpoint. As long as the results and the process are transparent and accountable, it could become extremely valuable in realising the value of comments individuals make when reacting online to events, people, and issues to inform decision-making, much like the complaint system I saw in the 1970s, but more powerful.

They call this “social listening”, but I like to refer to it as “social intelligence”. By any name, these platforms have a great deal of promise and a growing range of application. I would be most interested in other approaches to the development of such social intelligence.

References

Dutton, W. H. and Kraemer, K. L. (1978), ‘Management Utilization of Computers in American Local Governments,’ Communications of the ACM, 21 (3), 206-18.

Kraemer, K. L., Dutton, W. H., and Northrop, A. (1981), The Management of Information Systems, New York: Columbia University Press.