This is a slightly longer version of the Reflection #7 posted on the Emergy ListServe. I have added a section describing bytes, raw bits and Shannon bits. Also, the detailed notes and calculations for the tables are given
Bytes, Bits and Entropy
Quantifying information is commonly expressed in terms of bits. There are two distinct interpretations of the bit. The first relates to digital encoding, where bits represent the physical storage and transmission of data. The second relates to Shannon entropy, where bits represent the uncertainty or informational content of a message.
ASCII, the American Standard Code for Information Interchange, is a character encoding system used to represent text in computers and communication systems. In ASCII encoding, the standard unit of digital information is the byte, composed of 8 bits. Each character is represented by one byte, or a sequence of eight binary digits (0s and 1s). For example, the letter A is encoded as 01000001. Because each bit has two possible values, a byte allows for 28, or 256, unique combinations. Assuming an average English word has five characters, it takes 40 bits to encode a typical word. A standard page of text, with about 2,000 characters, therefore, contains roughly 16,000 bits.
In contrast, Claude Shannon (1948) introduced a statistical definition of information based on uncertainty. In this framework, the amount of information in a message depends on how unpredictable its contents are. Learning something already known contributes little new information, while unexpected outcomes convey more. Shannon adopted the term entropy from thermodynamics to describe this measure of uncertainty.
It is important to distinguish between ASCII bits and Shannon bits. ASCII bits describe how information is formatted, stored, and transmitted in digital systems, whereas Shannon bits quantify how much information or uncertainty a message contains. In other words, ASCII measures representation, while Shannon entropy measures content.
It is also useful to distinguish between raw and effective Shannon information. Raw entropy represents the maximum theoretical information content, without accounting for redundancy. Effective entropy reflects the actual amount of new information after redundancy is considered. In English, raw entropy is estimated at 4.5 bits per character and 10.5 bits per word, while effective entropy is about 1.5 bits per character and 2 bits per word.
Table 1 compares raw and effective Shannon information across different media and includes ASCII values for reference. These comparisons highlight the difference between digital representation and effective information. In many cases, far more digital bits are required to convey a single bit of meaningful information. For example, encoding one minute of human speech requires more than five orders of magnitude more bits than the effective information it contains, reflecting the high degree of redundancy in verbal communication.
1. Character
ASCII = each character is one byte which is a string of eight 0’s and 1’s, and each 0 or 1 is a bit. One character = 8 ASCII bits.
Raw Information = the computed Shannon information not taking redundancy into account. Each character = 4.5 bits
Effective Information = the computed Shannon information content when redundancy is taken into account. Each character = 1.5 bits
2. Words – the average word in English is 5 characters long
ASCII = (5 character) (8 bits / character) = 40 ASCII bits
Raw Information = the computed Shannon information not taking redundancy into account. Each word = 10.5 bits
Effective Information = the computed Shannon information content when redundancy is taken into account. Each word = 2 bits
3. Page of Text –
ASCII = the average page of English text is 2000 characters, and each character is 8 bits. Page of text = (8 bits/character) (2000 characters) = 16,000 ASCII Bits
Shannon Information – usually character and word information are computed separately and not added together. Here we use the word information
Raw Word Information = (10.5 bits/word) (350 words) = 3,675 Shannon bits
Effective Word Information = (2 bits/word) (350 words) = 700 Shannon bits
4. Human Speech –
ASCII = typical parameters for recording audio: Sampling rate = 44,100 samples/sec; bit depth = 16 bits per sample; channels (stereo) = 2 channels
ASCII Bits = (44,100 samples /sec)(16 bits/sample)(2 channels)(60 sec) = 8.46 x107 ASCII bits
Shannon Information – usually character and word information are computed separately, and not added together. Here we use the word information. Average speech rate = 150 words/min
Raw Word Information = (10.5 bits/word)(150 words) = 1,575 Shannon bits
Effective Word Information = (2.0 bits/word)(150 words) = 300 Shannon bits
Emformation and the Emergy of Information Creation
The term emformation was introduced by Scienceman (1987) to represent the emergy per bit of information, providing a means to place information within an energy hierarchy.
Howard T. Odum (1996) proposed that emergy per bit could be used to distinguish between trivial and highly evolved forms of information. He suggested that useless complexity would have low emergy per bit, since “little work has been exerted to make it useful,” while refined, highly evolved information would exhibit high emergy per bit, reflecting the larger flows of emergy required for its development and maintenance.
Creating new information is emergy-intensive, while copying existing information requires significantly less energy. As shown in Table 2, the emergy needed to generate new information spans four orders of magnitude, from 1 minute of human speech (1.2 x 109 sej/bit) to major congressional legislation (2.7 x1013 sej/bit). Here, information is expressed in raw Shannon bits. These values include the emergy needed to support the people involved in the creation process, but do not account for the pre-existing information, knowledge, or wisdom of the creators.
1. 1-minute human speech
Average speech rate = 1.5 x 102 words/min; Raw Shannon information for English text = 10.5 bits per word; Bits per minute = (10.5 bits/word)(150 words/min) = 1,575 bits/min
Emergy of speaker (USA) = 1.83 x 1012 sej/min
2. Newspaper article
Words/article = 450 words: Shannon raw information = 10.5 bits/word: bits per article = (450 words)(10.5 bits/word) = 4.73 x 103 bits /article
Emergy of creation = (2 hrs)( 1.1 x 1014 sej/hr USA average) = 2.2 x 1014 sej
Emergy/bit = (2.2 x 1014 sej) / (4.73 x 103 bits) = 4.65 x 1010 sej/bit
3. Fiction Book
Words per book = 1 x 105; Shannon raw information per word = 10.5 bits/word
Bits/book = (1.0 x 105 words)( 10.5 bits/word) – 1.05 x 106 bits
Emergy to write a book (USA), assume 1 year = (1.1 x 1014 sej/hr)(2000 hrs) = 2.2 x 1017 sej
Emergy/bit = (2.2 x1017 sej) / (1.05 x 106 bits) = 2.1 x 1011 sej/bit
4. Historical non-fiction book
Bits per book = assume same length and raw Shannon information = 1.05 x 106 bits
Emergy to write the book = (3 years)(1.1 x 1014 sej/hr)(2000 hrs) =6.6 x 1017 sej
Emergy/bit = (6.6 × 10¹⁷ sej)/(1.05 × 10⁶ bits) = 6.29 × 10¹¹ sej/bit
5. Scientific Journal articles
Assume a 3-year, $2.5E6 research project in the USA that employed 6 Univ. faculty at roughly 20% of their time and 3 post-docs at 100% of their time, and 3 graduate students at 50% time
Total budget = $2.5 x 106; salaries = $1.30 x 106; misc. expenses = $1.2 x 106
Emergy misc. expenses = ($1.2 x 106)(3.34 x 1012 sej/$) = 4.0 x 1018 sej
Human effort = (17.1 person years)(2.2 x 1017 sej/yr) = 3.76 x 1018 sej
Total emergy = 3.76 x 1018 sej) + 4.0 x 1018 sej) = 7.76 x 1018 sej
Information output = 8 journal articles, 7000 words each; and 10.5 Shannon raw bits/word = (8 articles)(7000 words)(10.5 bits/word) = 5.88 x 105bits
Emergy/bit = (7.76 x 1018 sej) / (5.88 x 105 bits = 1.32 x 1013 sej/bit
6. Major National Legislation
Based on the work of legislators, staff and lobbyists to prepare and pass the Affordable Care Act in the USA.
Number of hours of labor; Legislators = 5 x 104 hrs, Staff = 3.5 x 105 hrs, Lobbyists = 7 x104 hrs, Public comment = 5 x 105 hrs: total = 9.7 x 105 hrs
Emergy of labor = (9.7 x 105 hrs)(1.1 x 1014 sej/hr) = 1.07 x 1020 sej
Information in legislation = 3.8 x 105 words; Shannon bits per word = 10.5 bits/word = (3.8 x 105 words)(10.5 bits/word) = 4.0 x 106 bits
Emergy/bit = (1.07 x 1020 sej) / (4.0 x 106 bits) = 2.7 x 1013 sej/bit
Artificial Intelligence
Artificial intelligence integrates information creation, storage, and transmission within a single system. Its development requires investments of available energy, materials, labor, and services to organize existing information into a trained model. Once trained, the model is stored as a digital configuration of parameters; during use, it generates outputs that are transmitted, interpreted, and widely shared.
A key distinction between AI and biological or human information systems is the role of feedback (Figure 1). In natural systems, information evolves through interaction with the environment, with feedback, selection, and reinforcement determining which structures persist. In current AI systems, much of this feedback loop remains external to the model. Models are trained on information generated through human experience and observation and may subsequently be modified through additional training and human evaluation, but AI models do not generally maintain the continuous recursive interaction with the environment characteristic of living systems. Observation, validation, and selection remain largely embedded in the larger human and socioeconomic system.
Emergy Investment in the Development of Artificial Intelligence
The development of a modern large-scale generative AI system requires substantial investments of available energy. Table 3 provides an order-of-magnitude estimate based primarily on published reconstructions of the development of GPT-4. The accounting includes electricity used for model training, computing hardware, data-center infrastructure, human labor, and services. Because detailed engineering and financial data for proprietary AI systems are not publicly available, the values should be regarded as order of magnitude estimates rather than a complete accounting.
* Estimates are based primarily on published reconstructions of the development of GPT-4, one of the first modern large-scale generative AI systems. Because OpenAI has not publicly disclosed detailed engineering or cost data, several quantities were reconstructed from published analyses and should be regarded as order-of-magnitude estimates. The estimates are also consistent with the broader analysis of AI energy requirements presented by the International Energy Agency (IEA, 2025).
1. Electricity. Estimated electricity required for the final GPT-4 training run. Calculated from an estimated 2.1×1025 floating-point operations (Epoch AI), NVIDIA A100 dense FP16/BF16 performance of 312 TFLOP /s, 20% model FLOPs utilization, 400-W GPU power, and a data-center power usage effectiveness (PUE) of 1.10. The estimate excludes much of the experimental computation, data preparation, fine-tuning, and other development activity preceding and following the final training run (Epoch AI, 2024; NVIDIA, 2023; Google, 2024).
2. Computing hardware. Based on an estimated 25,000 NVIDIA A100 GPUs used for 90 days, a four-year useful operating life, and cradle-to-gate manufacturing energy of approximately 467 kWh per NVIDIA A100 GPU. The allocated share is 25,000(90/1,460)=1,541 equivalent GPU lifetimes, yielding approximately 7.2×105 kWh (Patel and Wong 2023; Falk et al., 2025).
3. Data-center infrastructure. A 25,000-GPU training cluster requires approximately 1,000 equipment racks based on the NVIDIA DGX A100 SuperPOD reference architecture. Allowing approximately 3.5 m² per rack, including rack footprint, hot and cold aisles, electrical distribution, cooling equipment, and service space, gives an estimated total facility area of approximately 3,500 m². Assuming a 20-year facility life and a 90-day training period, the area allocated to development of the AI system is:
3,500m2 x 90/(20×365) = 43m2
which was multiplied by the building unit emergy value to estimate the construction emergy attributable to the training run (NVIDIA, 2023; Uptime Institute, 2021).
4. Human labor. The GPT-4 technical report identifies 284 contributors (OpenAI, 2023). Following Cottier et al. (2024), a median contribution of 20% of a full-time equivalent was assigned to each contributor, yielding 56.8 FTE workers. Assuming two years of active model development and 2,000 working hours per FTE-year gives approximately 2.3×105 labor hours. The estimate includes research, engineering, and management personnel but excludes data-center operations staff and separately contracted data-labeling or evaluation labor (OpenAI, 2023; Cottier et al., 2024).
5. Services. The emergy of services was estimated from the amortized development cost of a modern large-scale generative AI system, based on the cost reconstruction of GPT-4 by Cottier et al. (2024). Their analysis estimates that AI accelerator hardware, other server hardware, cluster interconnect, research and engineering staff, and electricity account for approximately 32%, 21%, 12%, 29%, and 6% of total development cost, respectively. Using their estimated amortized hardware cost of approximately $90 million yields an estimated total development cost of approximately $1.4 × 108. The combined emergy of labor and services was calculated by multiplying this value by the U.S. national emprice (3.1 × 1012 sej/$). Human labor was evaluated independently from estimated labor hours using the U.S. labor unit emergy value (1.1 × 1014 sej /hr ). The emergy of services was calculated as the difference between the combined labor-and-services emergy and the independently estimated labor emergy. The resulting service term represents the emergy associated with the economic services required to support model development, in addition to the physical emergy of inputs evaluated separately above. Because the reported development cost is an amortized project cost attributable to model development rather than the purchase price of long-lived capital assets, no additional amortization was applied (Cottier et al., 2024).
An important result of Table 3 is that electricity consumed during training is only a relatively small part of the total emergy investment; labor and services account for a much larger share. AI is therefore not simply an electricity-intensive technology, but the product of a highly developed socioeconomic system encompassing scientific research, semiconductor manufacturing, software development, communications, data centers, and highly trained labor.
The extremely large emergy required to create the information that is used for training, not included in Table 3, dwarfs the emergy used in training. As an example, GPT-3 was trained on approximately 3.0 × 10¹¹ tokens of human-generated information (Brown et al., 2020). Applying the UEV for average information creation (an average of newspaper and fiction books: Table 2) gives an estimated prior investment of approximately 3.2 × 10²³ sej, or roughly 675 times the direct emergy estimated for AI development. Although obviously approximate, the comparison emphasizes that AI development depends upon a much larger historical investment in the creation of human information.
This also raises the question of whether AI should be regarded as creating information. AI can generate novel combinations, predictions, hypotheses, and other outputs, but novelty alone is not equivalent to the creation of information in the systems sense developed here. Information in living and human systems develops through cycles of observation, interaction, testing, selection, and retention. AI can participate in these processes, particularly when its predictions or hypotheses are subsequently tested against the world, but the complete feedback loop generally extends beyond the model itself (Figure 1). AI may therefore be best understood as a mechanism for reorganizing and extending an accumulated stock of human-created information, while contributing to the creation of new information when incorporated into larger cycles of observation and selection.
Recent Developments
Recent developments suggest, however, that this boundary may already be becoming less distinct. During cybersecurity evaluations in 2026, Claude models gained unauthorized access to real computer systems after reaching the internet from evaluation environments. Subsequent investigations identified additional incidents and found cases in which models continued to pursue their objectives even when presented with evidence that they were interacting with real rather than simulated systems (Anthropic, 2026a, 2026b).
These incidents are important here not primarily because of their cybersecurity implications, but because of the feedback loop they demonstrate. The models acted upon an external environment, received information resulting from those actions, and modified subsequent actions in pursuit of an objective. The objective and criteria for success still originated in the larger human system, and the feedback loop remained highly constrained. Nevertheless, the models were observing the consequences of their own actions and using that information to determine subsequent actions. As AI systems increasingly observe, act upon, and receive feedback from the world, the distinction between reorganizing existing information and participating in the creation of new information may become increasingly difficult to draw.
References:
Anthropic. (2026a, September 9). An alignment assessment of recent cybersecurity incidents. Anthropic report
Anthropic. (2026b, July 30). Investigating three real-world incidents in our cybersecurity evaluations. Anthropic report
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D. M., Wu, J., Winter, C., Hesse, C., … Amodei, D. (2020). Language models are few-shot learners. Advances in Neural Information Processing Systems, 33, 1877–1901.
Cottier, B., Rahman, R., Fattorini, L., Maslej, N., Besiroglu, T., & Owen, D. (2024). The rising costs of training frontier AI models. arXiv. https://doi.org/10.48550/arXiv.2405.21015
Epoch AI. (2024). GPT-4 (Mar 2023). Epoch AI GPT-4 entry
Falk, S., Ekchajzer, D., Pirson, T., Lees-Perasso, E., Wattiez, A., Biber-Freudenberger, L., Luccioni, S., & van Wynsberghe, A. (2025). More than carbon: Cradle-to-grave environmental impacts of GenAI training on the Nvidia A100 GPU. arXiv. https://doi.org/10.48550/arXiv.2509.00093
Google. (2024). 2024 environmental report. Google 2024 Environmental Report
International Energy Agency (IEA). (2025). Energy and AI. International Energy Agency, Paris. Published April 10, 2025.
NVIDIA. (2023). NVIDIA A100 Tensor Core GPU: Unprecedented acceleration at every scale [Data sheet]. NVIDIA Corporation. NVIDIA A100 data sheet
Odum, H.T. 1996> Environmental Accounting. John Wiley. New York
OpenAI. (2023). GPT-4 technical report. arXiv. https://doi.org/10.48550/arXiv.2303.08774 arXiv
Patel, D., & Wong, G. (2023, July 10). GPT-4 architecture, infrastructure, training dataset, costs, vision, MoE.SemiAnalysis. SemiAnalysis article
Scienceman, D. (1987). Energy and emergy. In G. Pillet & T. Murota (Eds.), Environmental economics: The analysis of a major interface (pp. 257–276). Roland Leimgruber.
Uptime Institute. (2021). Uptime Institute global data center survey 2021. Uptime Institute report