INFORMATION TECHNOLOGY AND DATA COLLECTIONS BOTH VIRTUAL AND PHYSICAL (part 1 of 4)

• Part 1 of 4

It is a natural process of the human mind to collect and organize sensory data and use it to develop adapted conceptualizations of the world around us. The discrete storage of this sensory data in the individual is limited, and the mind cannot always recall the stored data in an accurate way—details, perhaps minor but none-of-the-less important, become lost in the process. Humanity’s natural genus for languages was at first adequate to overcome the individual’s imperfections in memory, i.e. data storage. Communication allowed a sort of diversified and defused network repository, and as this common knowledge was recalled and transmitted among a close-knit group of people, say a prehistoric hunter-gather village, the collective knowhow of the community could act as a sort of checksum, if we were to borrow a similar term from computing, to verify the validity of the data as it spreads throughout the small human network. A less technical analogy would be that if prehistoric ‘Bob’ saw prehistoric ‘John,’ his neighbor, attempting to hitch his beast of burden behind his cart, Bob would quickly remind him of his error—the beast goes in front of the cart, for it will not function in the manner in which he currently has it—or perhaps denounce his lunatic neighbor to the local priestly-class as a spawn of the Great Evil so-and-so fit for the communal sacrifice. Much like today, neighbors throughout history have not always liked each other; I definitely do not care for mine and would perhaps welcome an opportunity to denounce them to a ‘local priestly-class for the communal sacrifice.’ Either way, the error would be identified by this common human networking of information, and through language and not writing, the memory would be rectified.

Given the fundamental importance spoken language plays in human society, our past counterparts’ ability to memorize was quite impressive in comparison to ours today, as most forms of storytelling and cultural transmissions were oral—writing and record-keeping being mostly the prevue of monastics, traders, and what little that passed for governments. As the millennia went by, and communities developed sophisticated technologies and skills from primitive divisions of labor, along with more diversified social orders to leverage these changes, it became necessary to support the basic human network with more permanent methods of record keeping other than the traditional oral transference. Cave drawings, pictographs, basic and universal symbols, sometimes leading to the establishment of basic writing systems, all became humanity’s way of creating information artifacts in aid of the individual’s memory abilities and data storage. As computer engineers, neuroscientist, and psychologist researching artificial intelligence in computers discovered, transmitting the comprehension of even the most basic human knowledge is extraordinarily complex in nature, and its recording requires creation and storage of many millions of information artifacts supported by billions of data artifacts. Where communities and civilizations developed writing systems, an abbreviated and virtual form of local inter-cultural communications, those information artifacts usually took the form of text stored into a solid medium such as papyrus, wax and clay tablets, animal skins, stone, wood, and in the case of the Chinese, paper. Collecting and preserving these information artifacts became extremely important to the community for various reasons. Communal lore masters and shamans turned into the default overseers of these information artifact collections as specialist scribes and copyist—as they usually played this role when the spoken word was the sole median and were the closest thing early man had to an ‘intellectual’ field. Through hundreds of generations of gradual social, political, and cultural development, men such as Plato and Aristotle, when developing a large and ecliptic collection of scrolls at the Academy and Lyceum near Athens in the 4th century B.C.E., or the royal record-keepers of the local lord/king developed into the very distant harbingers of the modern-day archivists, curators, accountants, and the various library professionals. Of course, the development of the field has not and will not stop with us. It will continue to evolve along with the societies it serves.

As the collation of information became more important to effective decision making in any comprehensive endeavor, most governments quickly realized the close relationship between information control and control over the general opinions of their populations. Ancient civilizations, as do the nation-states of today, expended a great deal of their time and resources in public perception and, by natural extension, its manipulation and control—finding it to be both cheaper and more effective than systematic violence and intimidation exercised by a large armed force. Soldiers are expensive as they are today, and mass liquidations in the name of the ‘public interest’ are messy and dubious. Besides, the wise tyrant of five millennia ago knew what the modern tyrants of today understand that no despot, no matter how unenlightened, has ever been overthrown by its propaganda ministry; whereas, the military and in rarer cases the people is always the primary rival for power and usually becomes the main cause of their regime's downfall. The People's Republic of China is one of many examples of this philosophy today where the basic education and news systems are stripped of any information exposure other than the government's approved ideology, and the extensive rewriting of histories and records that are not in-line with doctrine are purged from all public access of information. The United States government is of course no better in reality, but usually more refined in its methods. Granted, all societies and polities practice an overt and/or covert form of cultural indoctrination, often times usurped and directed by governing cliques and factions, but the Chinese have taken this practice to a high art form and have long been recognized as masters in this field even before the hegemony of the communist regime, particularly given the large scale population in which they have to operate over. Information control has always been an effective tool in controlling both individuals and groups of people, usually through ideologies and formative mythos. Information flow, whether truth or rumor, becomes the greater energizer of a socially connected body of people; control the information and you have a large amount of control over the peoples’ thoughts that they use to inform their political behavior.

A historical example is provided to us in the 13th through 15th centuries when the victorious Mongol armies raged across the Eurasia continent, conquering peoples from modern day Hungry to the Koreas and eventually leaving long-lasting dynasties in Russia (the Golden Horde), India (the Moguls), and China (Yuan Dynasty). Even though the diverse Asian and Turkic tribes of the Mongolian and central Asian steppes were predominately illiterate pastoral communities mixed with large contingents of settled and semi-settled peoples, they depended on the sophisticated governmental institutions of their conquered subjects—namely the Persian and Chinese—to run the complex bureaucracies of their polyglot empires. Dependent as they were on these loose foreign coalitions and patchwork governments for enlightened ideals to even rudimentary forms of public administration, the Mongol rulers always understood the power of information and its control in manipulating the perceptions of the masses they controlled and the many and diverse armies of the opponents they faced. In warfare, these warriors of the steppes rarely had the numerical advantage over their opponents and could not afford to suffer large causalities like their more settled counter-parts could, and siege craft was well beyond their technological developments short of the siege experts they dragooned from among their conquered subjects, so they relied heavily on terror and universal ignorance of their culture at the same time methodologically leveraging them in creative and diverse ways to demoralize and intimidated their opponents into submission or compliance. Terror and myth-making kept the Mongol prince in power in foreign lands. The chronicles of defeated opponents are filled with fear and loathing of the dreaded ‘Tatars’ and the 'vast' hordes in which they swept over their lands, like a plague, destroying everything in its wake. Modern studies show that Mongol armies were usually modest, especially in comparison to the large Chinese, Khwarezm, and Rus armies they faced and defeated in their many diverse theater of operations—rarely numbering over 30,000 horsemen per column of march—but they leveraged their speed, firepower, and maneuverability, along with denying the enemy useful intelligence on the composition and capabilities of their forces through a remarkably cleaver combination of disinformation and information control. 

During their empire era, after the death of Genghis Khan, the Mongol conquerors and their quisling administrators strictly controlled the access of information concerning the history of the Golden Family, i.e. the children and grandchildren of Genghis Khan who would become future khans and rulers of large parts of the empire as diverse appanages. The new rulers understood that such a humanization of their life and heritage that these documents would show could quickly reduce the aura of majesty surrounding them in the minds of their terrorized and subjected peoples; a fatal development given that they were always a conqueror minority nested among a much larger and usually far more sophisticated majority population. The Secret History of the Mongols, one of the few contemporary chronicles written on the life, fall, and rise of Genghis Khan and the beginnings of the first hordes before they sweep off the steppes to destroy and terrorize all the lands and people around them, could only be seen by members of the Golden Family and high level government officials. In reflection of these policies, very few court insiders such as the Grand Wazir Rashid al-Din had access to not only the Secret History but also the Altan Debter, an official Mongol history with a highly restricted circulation, for study. As he was writing his own great work, Jami’ al-Tavarikh (the Compendium of Chronicles) in the 13th century, he was able to utilize these sources (Lane, 2004). Many rulers before and many governments after the Mongol conquest have followed similar methodologies in information control with similar goals but with varied results.

 COLLECTION MANAGEMENT

Collection management, the glue that keeps this large amassed pile of information and content functional and useful as some sort of semi-cohesive whole—perhaps closer to the analogy in physics of the amorphous solid—has become an area of expertise of its own in the greater library field. With the increased sophistication of collections, especially after the integration of multimedia and virtual content within the traditional physical stacks and the strong growth of network access of information, the operations of collection management have become a layered process based on the formats composing the collection. In the past, a library's collection management staff mostly performed purchasing functions and was usually divorced from the stacks operations, a curious workflow perhaps informed more by a traditional bias between librarian and non-MLS library professional staff then any kind of operational effectiveness. Whatever the case, this paradigm is no longer valid, that is if it had ever been valid, for the library of today and tomorrow, and those institutions which adhere to the older model will face grave challenges to their growth and relevance as collections and information technology integration increasingly becomes the norm. With the ubiquitous nature of digital technologies in our lives, especially the role of the Internet, virtual collections have exploded in importance; although, it is not the traditional online databases such as the subscription based scholarly collections but general population centered, mega-collections such as YouTube, Google Books and its off-shot Google Scholar, massive peer-2-peer file sharing networks spanning over the entire Internet, and others that have claimed dominance. All this was made possible with advances in networking and computing being available in an age of information consumption, with users and information exchange dwarfing even the largest of physical collections.

Today's sophistication dictates that we assimilate all the critical functions that assure the viability of the collection under one comprehensive concept of ‘collection management.’ These individual processes not only have to be developed to a high sophistication on their own, but also be integrated into a comprehensive system complete with supporting technologies and guiding methodologies. In a diverse library collection, these core processes of collection management would be broken down into the following workflows coordinated together: content management, physical space management, and virtual space management. Management of a physical space encompasses its space management, collection layout, and the physical and efficient access to the stacks. Content management—this overarching purpose and vision of the entire collection is what some library professionals confuse as the sole component of collection management—is controlled through content selection including purchasing and weeding, quality control of the shelving order, and your institution's indexing system for information access. The elements of management most important to the virtual space management are data-portal access to the content, maintenance of the information technology infrastructure, and the database model and its management system used to store and manipulate the data in its electronic form.

Figure 1

A layered breakdown of the Collection Management processes and a few of their core elements.

Some areas the library field has done well in researching and documenting functions of the collection management process, but other areas such as comprehensive indexing systems, space management of the physical stacks, and concepts in virtual space access are a few core areas of collection management that remain extremely primitive practices in the library field. Libraries are still managing to organize their operations to reflect these realities, and a few of the core process of collection management do not have any suitable systems or technologies to handle specific processes in general, or the systems they implement are outdated and inefficient for today's user environment needs. Space management in particular is in a dismal state. In the case of space management of the stacks, there is almost no research or documentation to speak of let alone any kind of generalized system for library staff to implement. We produce many manuals on weeding collections, on content purchasing and content development, stacks presentation, and all of these are all very important and needed tools in the collection management tool bag, but the one tool of space management has always been missing. When we needed that tool, and we all eventually do, library staff and administrators have been forced to improvise in primitive and ineffectual ways—sort of like using our shoe as a make-shift hammer because we are too lazy to go out and buy a real hammer—only crudely effective in a limited extent. Given the importance of all these core elements, indexing, virtual space access, and virtual/physical space management, their primitive practices reflexes the poor state in how most institutions comprehensively manage their collections. All these short-coming soon become very visible to the library patron.

The following chapters will see new technologies introduced to handle some of these lesser developed core elements of many institution's collection management needs. Each proposed system has been research and developed with the NITA Methodology established to develop new technologies. The general idea is to develop the area of collection management to a level on par with more traditional, and highly developed, practices of the library field, such as circulation or information services—saving and utilizes what works and developing new paradigms for what does not work. Objectives such as fully efficient content access can only be achieved through proper collection space management augmented by an effective indexing organization, such as the KATIE Index presented in a later chapter. It is the same on the management of the virtual collection side. The efficient 'space' management implemented by a reliable and efficient database, such as proposed by the LISA Universal Informationbase System and its robust and unique data model in a following chapter, provided through a management information technology and infrastructure combined with the KATIE Index as an indexing system would make for truly effective access to the content of any collection, physical, virtual, or a hybrid of the two.

PHYSICAL COLLECTIONS EVOLVING INTO THE VIRTUAL

The physical collection has been the foundation of libraries and archives since the beginning of systematic collation of data and record keeping for the purpose of building knowledge-bases. These precursors to our modern day libraries and archives have been around for thousands of years, since the first societies of mankind started to organize their communities into complex and sophisticated hierarchies, and it became a necessity to keep records as a prerequisite in establishing systems of administration and governance, carrying out trade and warfare. With the need for record keeping, there also arose the need for a specialized class of record keepers or scribes with counterparts in the many different tongues of every major civilization; a few examples—

• Ancient Egypt: sesh
• Classical Greek: γραμματε?? / grammateus
• Sanskrit: ???? / lekhaka
• Hebrew: ?????? / sofér
• Middle Persian: dipir
• Old Chinese: 作册 / zuòcè, which is a verb–object compound literally "make-writing" (Sagart, 1998)

Along with the position of scribe, special repositories were established where these records could be kept in a manner determined by the accounting. These, in some cases very extensive, collections of recorders can be found in every major ancient civilization; whether they were transactions of trade, as in the case with most of the discoveries of the Linear A and B scripts of the Minoan and Mycenaean societies, tribute taking as with the records found for the Assyrians and Achaemenids of Persia and the Middle East, or observations on religious events, such as with the Mayan, Inca, and Egyptian empires, or documentation on the weather and harvest patterns so essential to the success of the common people. Some 30,000 clay tablets inscribed with Sumerian Cuneiform were discovered in regions of ancient Mesopotamia dating back to the 3rd millennium BCE (Krasner-Khait, 2001). There is also evidence of a substrate language of ancient comparable to the Sumerian located in the Indus river valley of modern day Pakistan and India dated to the contemporaneous Sarasvati-Sindhu civilization, circa 2500 BCE. Pertaining to the ancient Egyptian civilization, archaeologists have uncovered papyrus scrolls dated to around 1300-1200 BCE in the ancient cities of Amarna and Thebes.

Figure 2

Examples of the 4500 year old Indus script on seals and tablets from the Sarasvati-Sindhu civilization centered on the Indus river valley of modern day Pakistan and India. Image by Harappa Archaeological Research Project, J.M. Kenoyer, and the Dept. of Archaeology and Museums, Govt. of Pakistan (1995).

Figure 3

Information artifact of the first known written language, Cuneiform tablet impressed with cylinder seal: this one is of a trade transaction—in this case a receipt for glue. Dated back to the Ur III period, circa 2049 BCE, Mesopotamia, probably from Drehem (ancient Puzrish-Dagan). Image from the Sumerian collection at the Metropolitan Museum of Art, New York City (1941).

The ancient scripts of Linear A and Linear B correspond respectively to the two important and closely related civilizations on the island of Crete and the Greek mainland, called the Minoan and the Mycenaean. Linear A has been discovered on the island of Crete, which is strategically placed to take advantage of the trade lines between the sophisticated and literate societies of the Egyptians and Phoenicians at the Eastern end of the Mediterranean Sea. Though they had a strong presence on the Greek mainland as well, this island was the heart of the Minoan civilization. The script seems to have been used as a complete syllabary around 1900–1800 BCE, although several signs appear earlier as mason marks. Evidence shows that Linear A was used as the official script for the palaces and religious activities of the Minoan society and the contemporary Minoan hieroglyphs were mainly used on seals (Younger, 2000). As an undeciphered and lost language, the scripts that were discovered by Heinrich Schliemann on pottery from his Trojan excavation sites and inscriptions from central Crete, as well as a few similar potters' marks from Lahun, Egypt (12th dynasty), could have possibly come from an earlier period, circa 2100–1900 BCE, which coincides with the construction of these first Minoan palaces. Being an island people, the Minoans were seafarers and traders, so their interaction with the advanced Egyptians and Phoenicians were probably extensive—giving raise to the technological invention of the first known written languages on the European continent.

With the invasion, or perhaps rebellion, of the Mycenaeans from the rule of the Minoans, tablets inscribed with the Linear B script were discovered dating back as early as 1450 BCE in the palace archives of Knossos, Cydonia (Chaniotis, 1999), Pylos, Thebes and Mycenae. These writing systems, and writing in general, were to be lost with the fall of the Mycenaean civilization, suspected to be have been caused by a foreign invasion of peoples from the north around Macedonia and the Balkan region. The contemporary Egyptian and Assyrian powers make reference to a certain enigmatic ‘Sea People’ that might have been these foreign invaders who overran the Mycenaean Greeks as well as Crete and the Aegean islands. The classical Greek chronicles and stories would refer to them as the Dorians, especially in the works of Herodotus, which by that time had assimilated themselves among their native Greek subjects as one of four major Greek ethnē, or peoples of Hellas (Apollodorus, circa 200 CE). The Spartans would become one of the more famous examples of Dorian Greeks as well as the pseudo-Greek classical Macedonians and their close Epirots cousins. Whoever these foreign invaders or migrants were, Dorians, Illyrians, or Scythians, their conquest ushered in a period called the Dark Age of Greece. This was a mysterious historical period prior to the raise of the classical Greek polises and the rebirth of art, literature, and philosophy that the Roman and even our modern Western society owes much of its cultural and social foundations to—producing artifacts that we marvel at even today.

From ancient Greece, we turn back to the much older civilizations of Middle East and Asia Minor. Thousands of clay tablets were discovered in the palace of King Sennacherib, Assyrian ruler from 704-681 BCE, at his capital city of Nineveh, around modern day Kuyunjik, Iraq. More evidence turned up with the discovery of the personal collection of Sennacherib's grandson, King Ashurbanipal. The Royal Library of Ashurbanipal, the earliest formal library yet to be discovered, was excavated in the 1850’s, also in Nineveh. Its founder and namesake, King Ashurbanipal, was ruler of the Assyrian Empire from around 668-630 BCE (British Museum, 2002)—over three hundred years before the founding of the Library of Alexandria. In the classical world of the Mediterranean, the Great Library of Alexandria was seen as the archetype of learning and eruditions outside of the Athenian Academy, which went into decline under the rule of the Macedonian Diadochi. The Alexandrian library was founded around 331 BCE soon after the establishment of the city by the Macedonian conqueror Alexander II, the Great, and made prominent throughout the Mediterranean and Middle East under the rule of the first Macedonian pharaoh of Egypt, Ptolemy I Soter (305-282 BCE)—a general of Alexander's who seized the Egyptian territories of the Macedonian empire after the great conqueror’s death. The foreign Ptolemaic dynasty ruled the lands of Egypt for three centuries, terminating with the death of the infamous Cleopatra VII Philopator and the complete Roman annexation of this last remnant Diadochi empire by Augustus Caesar. It was a public library open to those with the proper scholarly and literary qualifications. The library benefited from the patronage of such famous peripatetics as Demetrius of Phalerum, as the library became a reflection of this school of philosophical thought, and held copies of the works of the inventor, mathematician, geographer, and astronomer Claudius Ptolemy (circa 90-168 CE). Pharaoh Ptolemy I and his successors wanted to understand the people under their rule as well as the world in general, so the library’s royal mandate and funding allowed the collection to house Latin, Buddhist, Persian, Hebrew, and Egyptian works—all translated into Greek (Krasner-Khait, 2001) for study by its patrons.

The Romans also founded and maintained many libraries throughout their empire, as most of their literary and philosophical thought was heavily influenced by the advanced, yet stagnating, Greek schools and achievements. Though no great lovers of learning themselves, the heavily Greek-influenced Macedonian conquerors of the Persian Empire carried the classical Greek philosophical concepts of erudition alone with them, and assimilated it with the oriental sophistication of the native Persian populations. With the 5th century fall of the Western Roman Empire and subsequent Medieval Age to follow caused by internal discord and foreign invasions of migratory barbarian tribes from the north and eastern steppes of the Black Sea region, some of the pursuits in the collection of knowledge were carried on by the Eastern sections of the empire, known as the Byzantine Empire, but the establishment of the Orthodox form of Christianity in intimate relations with the political elements of the Imperial administration assured that any collections of knowledge that could not be used to support the orthodoxy would be purged as heretical. As the Medieval Era and the Catholic Church established itself in the west, they followed a similar policy towards foreign ideas and control of information—assuring that any thought systems that could not be converted to support the faith would be ruthlessly and fatally purged as temptations for heretical thought. Repositories of knowledge were maintained in isolated pockets throughout Christendom, but these became treasured individual possessions and not for circulation. Though the universities and great Cathedrals, along with the great ruling dynasties of Western Europe and the Holy Roman Empire, kept libraries, they were small compared to their pre-Christian Greek and Roman versions and contemporary Islamic libraries and mostly followed the percepts of what the Church defined as appropriate study.

Ironically, it took the raise of Islam and its followers conquering Asia Minor, the Middle East, Egypt, North Africa, and most of the Iberian peninsula to preserve many of the great works from Greece and Rome, which were to be “rediscovered” by the Europeans in-depth when the strangle hold on knowledge by the Catholic Church was loosened with the illuminating sparks of the Renaissance. The magnificent centers of learning and their archives in Bagdad, Damascus, Cairo, and Cordoba allowed Muslim, and a handful of non-Muslim, scholars to study the collected works of the classical world as well as add unique inquiries of their own; especially in the fields of mathematics, medicine, and astronomy with a development comparable to the then contemporary societies of China and India. Europeans inherited from the Muslims a huge corpus of knowledge and a rediscovery of rationalism and the investigative approach to its development, an elaborate disciplinary architecture of knowledge, the notions of individual scholarship, and the idea of the college, all of which became central features of the European university exported to the rest of the world with the rise of European imperialism following the Age of Exploration (Zeleza, 2006). The drive for such academic discipline and curiosity was on the whole unknown to the Northern and Eastern agrarian-based tribal groups who took over the Western half of the Roman Empire and settled themselves as the modern peoples of Western and Mediterranean Europe today. Even the sophistication and advancement of the Romans leaned heavily towards the practicalities of governance, civil engineering, and the amazing logistics feats needed to subjugate and develop vast areas of land and control the millions of native peoples and societies within it. The Romans were not without their modest literary and cultural advancements, especially given that it was from the very beginning a heterogeneous empire and many of the middle and later Emperors were not even Latini or the greater Italian peoples, a curious event rarely seen in cultural empires of any period in history which are almost by definition fiercely racist. In comparison to the advanced societies they conquered, namely the Carthaginians, Greeks, Syrians, and Ptolemaic Egyptians, they were seen by these peoples as semi-barbarian and little better than the Germani, Gauls, and various other barbarian tribes to the North and East. In the Roman's defense though, the Greeks built their entire civilization on cultural snobbery, even calling the much more sophisticated Persians barbarians and were entirely mute on the Egyptians—who we knew through sources such as Plato, ironically thought of the Greeks as a young civilization (Jowett, 1971). The Romans were very proficient at assimilation and adaptation of cultural and philosophical advancements of those they interacted with, giving their advanced talent and energy in warfare, practical engineering, and cruel subjugation of foreign peoples a veneer of civilization.

• Continues in part 2 of 4. 

REFERENCES

Please see the last part.

Copyright Joseph Walker, 2020