Work and Heat
Let’s think of a caveman living beside a lake, who has just discovered fire. Energized by his discovery, he decides to test the waters. He takes a plunge in the lake and rubs the stones underneath the water. This will lead to an increase of internal energy of the stones and therefore raise their temperatures. As the temperature of the stones rise, the surrounding water will be heated up. As our protagonist is submerged in a big lake, the temperature of the total water does not see any appreciable rise. In other words, the water acts as a heat reservoir. He leaves the stones in water and swims back. The stones will eventually come in thermal equilibrium with the water (Zeroth Law). From basic energy conservation, then it is safe to say that nearly the entire work that we did on the stones can be converted into heat (First Law; no change in internal energy, assuming the initial temperature of the stones was the same as that of the water) that is dissipated into the lake. Is the reverse situation possible where we supply some heat to a system and convert the full heat into an equivalent amount of work? This question, thus related to the (19th century) studies on engines, lies at the heart of the Second Law of Thermodynamics.
The Second Law of Thermodynamics and Maxwell’s Demon
Maxwell writes in his book (Theory of Heat, 1872) that it is impossible, by the unaided action of natural processes, to transform any part of the heat of a body into mechanical work, except by allowing heat to pass from that body into another at a lower temperature.
Kelvin-Planck statement : It is impossible to construct an engine, that operating in a cycle, will produce no effect other the extraction of heat from a reservoir and the performance of an equivalent amount of work.
Clausius statement : It is impossible to construct a refrigerator that, operating in a cycle, will produce no effect other than the transfer of heat from a lower-temperature reservoir to a higher-temperature reservoir.
A key consequence of these statements is the idea that, the entropy of an isolated system undergoing a reversible process, cannot spontaneously decrease.
Maxwell hypothesizes that heat is related to the motion of body parts. Then if we can track, guide and control the parts of the body in motion, then by suitably arranging our apparatus we can transfer the energy of the moving parts of the heated body to any other body in the form of ordinary motion. If we can do this continually, then the heated body will become completely cold and an equivalent amount of work could be extracted. This process should then violate the Second Law but not the First Law. Maxwell claims that the Second Law is a denial of our abilities to perform such operations in any way yet discovered. He further says that the reason this is the case is because the motion of body parts related to heat must be so small that we cannot in any way lay hold of them to stop them.
Birth of the demon: Maxwell, in his correspondence to Peter Guthrie Tait, demonstrated through a thought experiment that the second law has a statistical limitation attached to it. Maxwell considers a box that contains a gas in thermal equilibrium and the temperature T is uniform throughout the volume of the box. Although the average speed is identical throughout the box, the speeds of individual molecules are random and follow the Maxwell-Boltzmann distribution.
He then imagines the box to be divided into two identical sections through the middle by a small door that is controlled by a finite being (later called a demon/daemon by Lord Kelvin). The demon, Maxwell imagines possesses the faculties that lets him follow the speeds of individual molecules. When a molecule on the left side of the box having a speed greater than the average speed approaches the door, the demon lets it pass through the door to the right side of the box. Similarly, when a molecule of speed less than the average speed approaches the door from the right side of the box, the demon lets it pass through the door to the left half. The demon keeps operating for a long enough time so that the right (left) side of the box progressively keeps getting hotter (colder).
We thus arrive at a situation where heat has been transferred from a colder reservoir to a hotter reservoir without the application of any external work. Equivalently, once a temperature difference has been set up between the two halves of the box, in principle, one can use the temperature gradient to extract useful work without spending any energy to build the engine in the first place. This is a direct violation of the two statements that we started the section with. Maxwell asserts in his book that this apparent violation of the Second Law is due to the statistical interpretation of systems containing large number of molecules, and individual molecules can in principle violate the Second Law. Thus according to Maxwell, the Second Law has a “statistical certainty”.
A crucial hidden assumption : Maxwell implicitly assumed that the measurement of the molecular speeds by the demon costs no energy. This assumption was questioned by Szilard in 1929, and as it turned out in later course of time, Maxwell’s Demon holds the door between Thermodynamics and the physics of Information.
Szilard’s engine and the significance of information: For the demon to not violate the Second Law, the mechanism of operation needs dissipation of energy.
Leo Szilard (1929) devised a thought experiment where a chamber of volume V contains a gas of a single molecule. First, a thin, massless, adiabatic partition is inserted into the chamber to divide it into two equal parts. The demon measures the position of the molecule and records the result for the next step. He then connects a load to that side of the partition where the molecule is supposed to be in from his previous measurement (i.e., the partition now acts as a piston). Keeping the chamber in continuous contact with a heat bath at a temperature T , the demon allows the gas to do some work by pushing against the partition. The work is done in a quasi-static, isothermal process, and therefore the internal energy of the gas remains unchanged. The process continues and the gas reaches its original state, i.e., having access to the full volume V , at which the partition is removed. During the expansion, heat Q is absorbed from the heat bath, entire of which gets converted into work W . Thus, Szilard’s engine completes a cycle where it extracts heat Q and converts it completely into W ; thereby apparently violating the second law. Szilard identified three key aspects related to the demon : measurement, information and memory.
As the gas expands isothermally, the pressure P exerted by the gas molecule on the partition follows an equation of state. For an ideal gas, one can write
P = (k_B T) / V .
Then the extracted work W is equal to k_B T log 2. Considering perfect conversion of heat into work, then the entropy reduction of the bath is given by
ΔS = k_B log 2.
If the second law has to hold, then this loss of entropy must be compensated for from somewhere. Szilard argued that the corresponding entropy production (i.e., increase of bath entropy) must be associated with the proces of measurement itself and the entropy thus generated must be at least equal to, if not greater than, the fundamental amount k_B log 2. Szilard’s engine ignited the philosophical link between thermodynamic entropy and a corresponding informatic one.
Erasure of information: Note that Szilard’s demon decides upon the side of the partition on which to attach the load from his last measurement, i.e., the initial position of the molecule is correlated with the side of the partition on which the load is to be attached. Now consider a situation where the gas molecule at time t = 0 is recorded to be on the left of the box. Correspondingly, the load is attached to the left side of the partition and the partition is then isothermally pushed to the right. There must certainly come a later time when the molecule will be on the right side of the box but the load will still be attached to the left side of the partition. This is an example of erasure of memory - or in other words, complete loss of correlation between the position of the molecule and the attaching of the load to a particular side.
Landauer’s erasure principle (1961) : Logical irreversibility and entropy production
Landauer and Bennett shifted the focus from obtaining information to erasing it: while in principle, there might be no entropy cost to gather information, there must be a price paid to destroy it.
According to Rolf Landauer, a device (computer) is logically irreversible if its output does not define the input(s) uniquely. Logical irreversibility, he argued, implies physical irreversibility, which in turn is accompanied by dissipation.
Let us imagine a computer with a finite N number of bits. We consider a RESTORE
TO ONE (RTO) operation where one performs some work to restore each binary bit to 1 from a thermalized initial state (any of the 2 N possible arbitrary sequences of 0s and 1s). The RTO operation resembles a logically irreversible process if the system forgets about its initial configuration : any restored bit could have been either 1 or 0 state initially, and after retoration does not carry any information of the pre-restoration state. A logically reversible computer would require storing information about every bit at every time step for all operations carried out on the assembly of bits. Such a machine would require an immense amount of storage space, and it is impractical to consider such systems in the real world.
With RTO (logical irreversibility), then the total entropy reduction in such a system of bits should be k_B N log 2. Since the entropy of a closed system (computer + its batteries) cannot decrease, then the entropy decrease in the system of bits must necessarily be compensated as dissipation (physical irreversibility) into the surroundings of at least an equivalent amount of heat Q = k_B N T log 2. Thus, erasure of memory is accompanied by entropy production into the environment by the system.
Energetic cost of erasure : Previously, we considered the case of erasure of memory in the Szilard engine as the gas performed isothermal work on the partition. Equivalently, we can consider the opposite situation where the gas molecule is restored to a particular (Left or Right) “initial” state by the application of a minimum amount of work W erasure . Charles Bennett argued that for the demon to observe a molecule, it must first forget its previous observation.
This costs energy and increases entropy.
Let the gas molecule be in a chamber of volume V . The demon stores the information by measuring the position of the molecule. It could be in either the left (L) or right (R) half of the chamber. To erase the information, we remove the partition from the middle of the box. Then we apply a piston at the right end of the box and push it inwards isothermally at temperature T until the compressed volume is V /2. The piston is then removed. The resulting state of the molecule is always L, regardless of which part of the box it was originally measured to be in. The work needed to carry out this memory erasure is then equal to k_B T log 2.
Smoluchowski, pressure demon and Brownian motion
Marian von Smoluchowski (1912) tackled the paradox through Brownian motion of the door. He considers that the door (an automated pressure demon) is connected to a spring and opens only in one direction, thereby allowing particles to accumulate in one half of the box. Then using the pressure difference, one can extract useful work. However, he argued that the door must be thermalized and have a Brownian motion of its own. This will allow the door to open even for molecules going in the opposite direction as any pressure difference arises within the two halves of the box. For the process to be cyclic, thermal energy must be removed from the door and this will lead to entropy production into the surroundings.
Norton and conservation of phase space volume:
John D. Norton attempts a solution to the paradox without taking the information route. In a nutshell, he claims that if we consider the gas and the demon together as our system, then for this coupled entity the functioning of the demon in the way Maxwell described will violate Liouville’s theorem. For systems described by equilibrium statistical physics, time evolution is governed by Hamilton’s equations. Liouville’s theorem states that the phase-space volume of any Hamiltonian system should be conserved.
Coupling a Maxwell’s demon (and its support systems) to the imagined target system in thermal equilibrium is equivalent to creating a larger system which has a larger phase-space than the imagined system. If the demon acts as Maxwell described, on the target system then the target system is taken out of its equilibrium configuration by the demon. Restoring the demon back to its own original state then effectively reduces the available space-space for the combined system from the full phase-space that would have been available to the combined system had it been in equilibrium. Now assuming that both the demon and the target system can be described by Hamiltonian dynamics, this is a violation of Liouville’s theorem and is not allowed in equilibrium statistical physics. Hence within the scope of equilibrium physics, the demon cannot function as intended. But is it correct to assume that in a coupled system we can separately restore parts to original states through Hamiltonian dynamics?
Moral of the story
The system of demon plus the box does not violate the second law of thermodynamics. Informatic entropy encodes the least fundamental contribution to thermodynamic entropy. Maxwell’s demon brought the focus on dissipation – an irreversible and unavoidable fact of nature. Modern statistical physics deals with active systems which are by nature out of equilibrium. A lot of current research focusses on the dissipation associated with maintaining active systems in steady states.
