I. GENERAL MOTIVATION
At a molecular level, diffusion of substrates and macromolecules combined with conformational changes of proteins underlie most biologic processes. Even though these molecular events are not occurring in thermodynamic equilibrium, they are actually in most cases not far from equilibrium. In general, processes that occur close to thermodynamic equilibrium are energetically favorable compared with processes far from equilibrium. Indeed, energetic efficiency is one of the major drivers of evolution (1). An instructive example is the energy consumption of the human brain compared with that of humanmade computers. The brain's computational power has been estimated to be above the most potent presently available supercomputers (2).
Despite this capacity, the energy demand of the human brain corresponds to only roughly 20 W, i.e., less than a conventional (oldfashioned) light bulb. In contrast, a typical supercomputer consumes on the order of megawatts, i.e., many orders of magnitude more. One of the reasons behind the incredible efficiency of the brain is that the mechanisms of biologic data processing operate rather close to thermodynamic equilibrium. The free energy differences between the various conformations of proteins are of the order of a few kT. The free energy corresponding to 1 kT is the energy that can be easily surmounted by random movements in thermal equilibrium. In terms of equivalent electrical voltage, kT corresponds to ∼0.025 V = 25 mV (i.e., kT/q_{e} ∼ 25 mV at physiologic temperatures, where q_{e} is the absolute electrical charge of an electron). This is a much smaller value than the operating voltage of transistors in computers, which is of the order of ∼3,000 mV = 120 kT/q_{e} . Because the probability of a reaction depends in an exponential manner on the free energy difference (between initial and final states), a value of 120 kT assures almost 100% deterministic operation, but on the downside, it is paid for with a large power consumption. In contrast, biologic processes are mostly not deterministic but often rely on redundancy to assure approximate accuracy. There are, of course, many exceptions to this simplified view. For example, for voltagegated cation channels, whose opening transition is accompanied by the translocation of up to 13 elementary charges, the free energy difference upon a potential shift of 75 mV will amount to 75 mV × 13 q_{e} , corresponding to 39 kT.
Even though, theoretically, proteins can exist in a practically unlimited number of conformations (or microstates), in many cases their enzymatic function can be described in terms of a limited number of states that correspond to an ensemble of microstates with similar conformation. A classic example is an enzyme that binds and transform a single substrate (Fig 1A). The cartoon in Figure 1A is clearly simplified, but it could provide more than a qualitative description of enzyme function. Indeed, the scheme of Figure 1A could be used, in principle, to generate a precise mathematic description of how the enzymatic activity depends on the concentration of substrates, conformational changes of the enzyme, and other factors.
A problem is that most biophysical and biochemical methods yield information on the average behavior of the ensemble of a large number of identical or similar molecules but do not capture the behavior of single molecules. In 1976, the patchclamp technique and bilayer recordings allowed one for the first time to observe the function of a single biologic molecule in real time (3, 4). Figure 1B shows a typical singlechannel recording from skeletal muscle: the openings and closings of a single acetylcholine receptor channel in the presence of 50 nM of suberyldicholine are clearly visible (5). Since these early experiments, the patchclamp method was quickly adopted in many laboratories worldwide and led to the discovery of a plethora of different ion channels present in practically all cell types (6).
Detailed analysis of singlechannel recordings revealed 2 important properties: the channels switch between distinct conductance states in a very short time (too short to be measurable by the patchclamp technique) and the distribution of the dwell times in these conductance states can be very well described by exponential distributions. This is illustrated in Figure 1C for a hypothetical ion channel with 2 closed states and 1 open state. The histogram of all openchannel events is well described by a single exponential distribution, whereas the histogram of the closedchannel events is wellfitted by a double exponential function (see pink curves in Fig 1C). In fact, very often the dwelltime distribution of shut (not conducting) periods shows a complex multiexponential distribution, originating from conformational changes occurring before channel opening. These conformational changes cannot be detected by changes of the current amplitude but can be indirectly inferred from the kinetic analysis. As will be explained in more detail in the following, these properties are very well described by a socalled time homogeneous Markov process. The defining feature of a Markov process is the lack of memory: the probability that a state transition (e.g., from closed to open) happens does not depend on how much time the channel has spent in the previous conformation. This concept will be explained in more detail later.
These rather general considerations explain why to obtain a mechanistic insight into a complex biologic process. It is essential to study the individual components of the system in isolation and under controlled conditions that bring them close to thermodynamic equilibrium. A prominent example is the propagation of electrical signals along the sometimes extremely long processes of nerve cells (axons), which has attracted considerable attention among physiologists in the past century. In particular, the studies of Hodgkin and Huxley on the basis of the generation and propagation of the neuronal “action potential” (6, 7) have paved the way for a detailed, mechanistic investigation of the molecular basis of electrical excitability. The action potential is an explosive change of the cellular membrane potential from a normally negative value (around −70 mV) to a positive value (around 40 mV, a process called depolarization) within roughly a millisecond, followed by a slightly slower repolarization of the membrane potential to the resting value of −70 mV. In nerve axons, that connect, for example, the brain to motor neurons and drive muscle contraction, the action potential travels as an excitatory wave at high speed toward the target neuron or muscle. The mechanisms underlying action potential generation and propagation have been described in terms of probabilistic discrete state continuoustime (i.e., Markov) models in a groundbreaking publication by Hodgkin and Huxley in 1952 (7). Intriguingly, this approach allowed for the laying out the mechanistic underpinning of the action potential 50 years before the threedimensional structure of the ion channels involved was available (8, 9). The neuronal action potential is driven by the opening of ion channels in the nerve membrane. Voltagegated Na^{+} channels underlie the initial depolarization. The reason for the depolarization is that the extracellular Na^{+} concentration is much higher than the intracellular Na^{+} concentration. Therefore, opening of Na^{+} channels leads to a Na^{+} influx at voltages more negative than the Na^{+} equilibrium potential, which is around +50 mV, i.e., much higher than normal membrane potentials. After the upstroke of the action potential, the combined effect of Na^{+} channel inactivation, presence of background K^{+} channels, and the delayed opening of voltagegated K^{+} channels results in the repolarization of the membrane potential. The reason why K^{+} channels repolarize the membrane potential is that extracellular K^{+} is much lower than intracellular K^{+}. Therefore, at voltages more positive than about −80 mV, the K^{+} equilibrium voltage in most cells, K^{+} efflux occurs upon opening of K^{+} channels.
On a historical note, Hodgkin and Huxley were not aware of the mechanistic value of the theory in terms of conformational change of proteins. Yet, the idea of probabilistic gates regulating opening of the voltagegated ion channels continues to be highly relevant for the understanding of ion channel and enzyme function in general.
Typical schemes of Markov models used to describe channel gating and transporter function are illustrated schematically in Figure 1D, which shows the Markov models discussed in the present article. The various states of the model are connected by rate constants (indicated by arrows), whose meaning will be defined in detail in the next section. The general mathematic treatment of such Markov models has been covered in several excellent publications (10), which are highly recommended for the interested reader. For a basic usage of MarkovEditor (http://users.ge.ibf.cnr.it/pusch/sframes/StateEditor.html), such deep knowledge is not strictly necessary but is certainly useful. In mathematic terms, Markovian systems respond in a relatively simple manner to an external perturbation, as, for example, a change in membrane potential or ligand concentration. The timedependent relaxation of the occupation probabilities of the various states upon perturbation of a Markov model is provided by the sum of n exponential functions, where n=Number of states 1, and where the time constants of the exponentials are the eigenvalues of the socalled Q matrix of the rate constants (10–12). For example, the relaxation of a 3state voltagegated channel upon a voltage jump is described by a doubleexponential function. Most electrophysiologists know very well that it is extremely difficult to extract more than 2 timeconstants from a given dataset. Thus, it may seem that Markov models with more than 3 states might be rather useless in practical terms. Nevertheless, a priori knowledge of the possible physical states of the system, combined with symmetry arguments, can provide deep insight into the molecular machinery of the channel or transporter of interest. Moreover, due to the mathematic complexity of the problem, for systems with more than 2 states, it is very difficult to understand at the intuitive level how changes of any rate constant affect the macroscopic properties.
II. TARGET AUDIENCE AND OUTLINE OF THE ARTICLE
The present article aims to provide a rather intuitive tool, accessible to a wide readership, to explore simple and complex Markov models, allowing to directly simulate the effect on channel or transporter behavior caused by changes of any microscopic parameter, both at the singlechannel level, as well as at the ensemble level. The article is organized such that the different applications of the computational tool MarkovEditor are presented as blocks of increasing complexity. This is done in different, selfcontained paragraphs such that the reader can choose the sections with the appropriate level.
Our idea is that this software and the relative tutorials are most useful for people who already have some background knowledge on ion channels or transporters and who desire a tool to be guided into the quantitative study of various specific aspects. A deep understanding of singlechannel behavior might be more difficult to achieve based solely on the more commonly studied properties of macroscopic currents explained, for example, in a university physiology course. The expected users of MarkovEditor are indeed master's students, master's student interns, PhD students or early postdoctoral students rather than complete beginners such as bachelor students. The concept is to help them visualize how the theory, summarized in equations, can actually describe a channel's behavior and what is the impact of some parameters, such as a rate constant, on the final current. This can be seen both at the single channel and the wholecell level. We envision some teachingscenarios, in which such a tool can be useful. A possible situation is when a principal investigator (PI) wishes to introduce into the world of Markov models PhD students or early postdoctoral students, who have worked only with wholecell currents. The PI, or more experienced members of the lab crew, could be available for guidance, when required. The software can also be used by professors teaching in advanced courses: the lecturer could go through the theoretic aspects step by step, while showing models and simulations with MarkovEditor. In this case, the software and the present article can also be used for home exercises, to practice on concepts introduced during class. Another situation in which MarkovEditor might be helpful are practical courses, in which a group of students rotate in a lab, where singlechannel experiments are routinely performed. MarkovEditor could be used to explain what they are going to see in the following weeks and what is the meaning of the data analysis they will be required to do. Finally, MarkovEditor can be a useful tool for active scientists working on transporters or channels or are using other singlemolecule approaches.
We should not forget to mention that other Markov modeling software is freely available. The most prominent are QuB (https://qub.mandelics.com/) and programs from the HJCFit suite (https://dcprogs.github.io/HJCFITdevdocs/). The scope of these programs is, however, quite different from that of MarkovEditor. HJCFit programs are highly specialized for professional singlechannel data analysis, including missed events correction (13). QuB has overlap with MarkovEditor but lacks several features, such as transport cycle analysis, varying parameters, and others. However, the reader should consult the corresponding Web sites to find out more detail on these programs.
III. THE 2STATE MODEL
Before delving into complex models, we will discuss the simplest possible Markov model, the 2state system (Fig 2A,B). We will first describe the fundamental mathematics of the system, defining the concept of rate constants and highlighting the essential feature on any Markovian system: the lack of memory. We will then use the 2state system to introduce most features of the MarkovEditor program.
The 2state system is schematically depicted in this (apparently innocent) scheme: \(\def\upalpha{\unicode[Times]{x3B1}}\)\(\def\upbeta{\unicode[Times]{x3B2}}\)\(\def\upgamma{\unicode[Times]{x3B3}}\)\(\def\updelta{\unicode[Times]{x3B4}}\)\(\def\upvarepsilon{\unicode[Times]{x3B5}}\)\(\def\upzeta{\unicode[Times]{x3B6}}\)\(\def\upeta{\unicode[Times]{x3B7}}\)\(\def\uptheta{\unicode[Times]{x3B8}}\)\(\def\upiota{\unicode[Times]{x3B9}}\)\(\def\upkappa{\unicode[Times]{x3BA}}\)\(\def\uplambda{\unicode[Times]{x3BB}}\)\(\def\upmu{\unicode[Times]{x3BC}}\)\(\def\upnu{\unicode[Times]{x3BD}}\)\(\def\upxi{\unicode[Times]{x3BE}}\)\(\def\upomicron{\unicode[Times]{x3BF}}\)\(\def\uppi{\unicode[Times]{x3C0}}\)\(\def\uprho{\unicode[Times]{x3C1}}\)\(\def\upsigma{\unicode[Times]{x3C3}}\)\(\def\uptau{\unicode[Times]{x3C4}}\)\(\def\upupsilon{\unicode[Times]{x3C5}}\)\(\def\upphi{\unicode[Times]{x3C6}}\)\(\def\upchi{\unicode[Times]{x3C7}}\)\(\def\uppsy{\unicode[Times]{x3C8}}\)\(\def\upomega{\unicode[Times]{x3C9}}\)\(\def\bialpha{\boldsymbol{\alpha}}\)\(\def\bibeta{\boldsymbol{\beta}}\)\(\def\bigamma{\boldsymbol{\gamma}}\)\(\def\bidelta{\boldsymbol{\delta}}\)\(\def\bivarepsilon{\boldsymbol{\varepsilon}}\)\(\def\bizeta{\boldsymbol{\zeta}}\)\(\def\bieta{\boldsymbol{\eta}}\)\(\def\bitheta{\boldsymbol{\theta}}\)\(\def\biiota{\boldsymbol{\iota}}\)\(\def\bikappa{\boldsymbol{\kappa}}\)\(\def\bilambda{\boldsymbol{\lambda}}\)\(\def\bimu{\boldsymbol{\mu}}\)\(\def\binu{\boldsymbol{\nu}}\)\(\def\bixi{\boldsymbol{\xi}}\)\(\def\biomicron{\boldsymbol{\micron}}\)\(\def\bipi{\boldsymbol{\pi}}\)\(\def\birho{\boldsymbol{\rho}}\)\(\def\bisigma{\boldsymbol{\sigma}}\)\(\def\bitau{\boldsymbol{\tau}}\)\(\def\biupsilon{\boldsymbol{\upsilon}}\)\(\def\biphi{\boldsymbol{\phi}}\)\(\def\bichi{\boldsymbol{\chi}}\)\(\def\bipsy{\boldsymbol{\psy}}\)\(\def\biomega{\boldsymbol{\omega}}\)\(\def\bupalpha{\bf{\alpha}}\)\(\def\bupbeta{\bf{\beta}}\)\(\def\bupgamma{\bf{\gamma}}\)\(\def\bupdelta{\bf{\delta}}\)\(\def\bupvarepsilon{\bf{\varepsilon}}\)\(\def\bupzeta{\bf{\zeta}}\)\(\def\bupeta{\bf{\eta}}\)\(\def\buptheta{\bf{\theta}}\)\(\def\bupiota{\bf{\iota}}\)\(\def\bupkappa{\bf{\kappa}}\)\(\def\buplambda{\bf{\lambda}}\)\(\def\bupmu{\bf{\mu}}\)\(\def\bupnu{\bf{\nu}}\)\(\def\bupxi{\bf{\xi}}\)\(\def\bupomicron{\bf{\micron}}\)\(\def\buppi{\bf{\pi}}\)\(\def\buprho{\bf{\rho}}\)\(\def\bupsigma{\bf{\sigma}}\)\(\def\buptau{\bf{\tau}}\)\(\def\bupupsilon{\bf{\upsilon}}\)\(\def\bupphi{\bf{\phi}}\)\(\def\bupchi{\bf{\chi}}\)\(\def\buppsy{\bf{\psy}}\)\(\def\bupomega{\bf{\omega}}\)\(\def\bGamma{\bf{\Gamma}}\)\(\def\bDelta{\bf{\Delta}}\)\(\def\bTheta{\bf{\Theta}}\)\(\def\bLambda{\bf{\Lambda}}\)\(\def\bXi{\bf{\Xi}}\)\(\def\bPi{\bf{\Pi}}\)\(\def\bSigma{\bf{\Sigma}}\)\(\def\bPhi{\bf{\Phi}}\)\(\def\bPsi{\bf{\Psi}}\)\(\def\bOmega{\bf{\Omega}}\)\begin{equation}\tag{1}C\matrix{ {\mathop \to \limits^\alpha } \cr {\mathop \leftarrow \limits_\beta } \cr } O\end{equation}
At any given thermodynamic condition, defined by membrane voltage, pressure, temperature, or concentration of ligands, this 2state system is characterized by 2 numbers: the opening rate constant α and the closing rate constant β (Fig 2A). The basic assumptions underlying this simplified description are

the channel can be either in state O (open) or in state C (closed; i.e., there are no intermediate configurations),

transitions between states occur abruptly, and

the channel is completely memoryless.
The latter assumption is what defines Markov systems and means that after a transition to a new state has occurred, the probability that a further transition takes place, is independent of how much time the channel has already spent in the current state. Only with this assumption the following definition of the rate constants α and β makes sense: Given a very small time interval dt and given that the channel is in state C at time t, the probability that it makes a transition to state O between t and t + dt is equal to \begin{equation}\tag{2}P\left( {O,t + dtC,t} \right) = \alpha dt\end{equation}
Here, the notation of conditional probability is used, i.e., P(AB) means probability of A, given that B is true. Thus, P(O, t + dtC, t) is the probability to find the channel in state O at time t + dt, knowing that it was in state C at time t. The probability of this transition is proportional to the time lag dt (as long as it remains very small) and proportional to the opening rate constant α. Thus, α provides the speed with which the channel switches to state O.
Similarly, the closing rate constant β is defined by \begin{equation}\tag{3}P\left( {C,t + dtO,t} \right) = \beta dt\end{equation}
These equations are valid for infinitesimal times dt. To deal with longer times, we will need the concept of a differential equation as described below.
The Markov property of lack of memory implies that the transition probability is independent of t and can thus be expressed as \begin{equation}\tag{4}P\left( {O,t + dtC,t} \right) = P\left( {O,dtC,0} \right) = \alpha dt\end{equation} i.e., knowing that the channel is closed at time 0, the probability to find it open after time dt is equal to αdt.
The opening and closing rate constants define the rate at which the system can surmount the energy barrier between the 2 states, as illustrated in Figure 2B. One can envision the system undergoing thermal fluctuations, attempting continuously to overcome the barrier characterized by a certain activation energy, E_{a} . Only occasionally has the system enough energy to surmount the barrier. This probability depends in an exponential manner on the activation energy (14). Most importantly, the transition probability is independent of how long the channel has already been in a given state.
These mathematic properties imply that P_{O} (t) and P_{C} (t), i.e., the timedependent state probabilities, obey the following fundamental linear differential equations: \begin{equation}\tag{5}{{d{P_O}\left( t \right)} \over {dt}} =  \alpha {P_O}\left( t \right) + \beta {P_C}\left( t \right)\end{equation} and \begin{equation}\tag{6}{{d{P_C}\left( t \right)} \over {dt}} = \alpha {P_O}\left( t \right)  \beta {P_C}\left( t \right)\end{equation}
Differential equations as these relate the rate of change of a quantity (i.e., its temporal derivative) to the value of the quantity itself. The previously mentioned equations are in essence equivalent to those used to describe chemical reactions by the law of mass action. For example, the isomerization of a compound from a cis to a trans conformation would be described by a completely equivalent set of differential equations.
Note that summing Eqs. 5 and 6 yields \begin{equation}\tag{7}{{d{P_O}\left( t \right)} \over {dt}} + {{d{P_C}\left( t \right)} \over {dt}} = 0\end{equation} ensuring that the total probability to be in any state remains constant and equal to 1.
In fact, for the 2state model \begin{equation}\tag{8}{P_O}\left( t \right) + {P_C}\left( t \right) = 1\end{equation} i.e., the channel has only 2 possibilities, to be either in the closed or open state. Thus, P_{C} (t) can be substituted by \begin{equation}\tag{9}{P_C}\left( t \right) = 1  {P_O}\left( t \right)\end{equation}
Inserting this substitution into (Eq. 5), i.e., replacing P_{C} (t) by 1 − P_{O} (t), leads to the following linear differential equation of the single function P_{O} (t): \begin{equation}\tag{10}{{d{P_O}\left( t \right)} \over {dt}} =  (\alpha + \beta ){P_O}\left( t \right) + \beta \end{equation}
This differential equation determines the response of the 2state system to a perturbation, which could be a change in voltage or a change in the concentration of a ligand. Upon the perturbation, the system relaxes to a new steady state, determined by the altered opening or closing rate constants, according to the fundamental timedependent singleexponential solution \begin{equation}\tag{11}{P_O}\left( t \right) = {\alpha \over {\alpha + \beta }} + \left( {P_O^0  {\alpha \over {\alpha + \beta }}} \right){e^{  \left( {\alpha + \beta } \right)t}}\end{equation}
Here, \(P_O^0\) is the initial open probability at time 0 (assumed to be known). It is instructive to verify that this is indeed a solution of the differential equation (Eq. 10), with the initial condition P_{O} (0) = \(P_O^0\) . At infinite time, which corresponds to a true equilibrium in this case, P_{O} reaches the steadystate value \begin{equation}\tag{12}{P_O}\left( \infty \right) = {\alpha \over {\alpha + \beta }}\end{equation}
The relaxation from the initial state to the final equilibrium occurs with the time constant \begin{equation}\tag{13}\tau = {1 \over {\alpha + \beta }}\end{equation}
Thus, using a slightly different notation, the fundamental solution (Eq. 11) can also be written as \begin{equation}\tag{14}{P_O}\left( t \right) = {P_O}\left( \infty \right) + \left( {P_O^0  {P_O}\left( \infty \right)} \right){e^{  {t \over \tau }}}\end{equation} and similarly one obtains \begin{equation}\tag{15}{P_C}\left( t \right) = {P_C}\left( \infty \right) + \left( {P_C^0  {P_C}\left( \infty \right)} \right){e^{  {t \over \tau }}}\end{equation}
Before inserting this 2state model into the MarkovEditor program for further exploration, we will endow it with a typical voltage dependence of the rate constants, a property that is fundamental for electrical excitation in neurons but also important in many electrogenic transporters. In voltagegated cation channels, highly specialized electrically charged segments of the channel proteins, the socalled voltage sensors, move within the electric field across the membrane upon a change of the membrane voltage (6). In the simple 2state model, voltage sensor movement and channel opening occur simultaneously, i.e., they are perfectly coupled. This is not fully realistic but useful to illustrate the basic concept. As depicted in Figure 2C, the gating charge q, associated with voltage sensor movement, can be either close to the inside of the membrane (in the closed state) or close to the outside (in the open state), providing stabilization of the open state at positive membrane voltages. It is convenient to express the gating charge of a single channel in terms of the absolute elementary charge of an electron, q_{e} = 1.602 × 10^{−19} C, i.e., \begin{equation}\tag{16}q = z{q_e}\end{equation} introducing the gating valence z. The rate constants depend in an exponential manner on the membrane potential V and on the gating charge in the following manner \begin{equation}\tag{17}\alpha \left( V \right) = {\alpha _0}{e^{\delta zV{q_e}/\left( {kT} \right)}}\end{equation} and \begin{equation}\tag{18}\beta \left( V \right) = {\beta _0}{e^{  \left( {1\;  \;\delta } \right)zV{q_e}/\left( {kT} \right)}}\end{equation}
These equations are crucial for all models discussed in this article: α _{0} and β _{0} are the respective opening and closing rate constants at a membrane potential of 0 mV, k is the Boltzmann constant, q_{e} the (absolute) charge of an electron, and T the absolute temperature. The relative apparent electrical distance δ takes into account a possible asymmetric dependence of the rate constants on voltage (see Fig 2C). For a more elementary introduction into the concept of electrical distance, the reader is referred to (6). The exponential voltage dependence of the rate constants is based on transition state theory, which cannot be explained in detail here; see (14). Briefly, as explained previously, the speed of a conformational change is determined by the activation energy. In addition to the conformational energy (expressed in terms of α _{0} and β _{0}), for a voltagedependent transition, the activation energy includes the energy that has to be provided to move the gating charge along a certain fraction of the electric field across the membrane (see Fig 2C). For opening transition, this fraction is given by δ and for the closing transition by 1 − δ. Note that Figure 2C makes the simplifying assumption that the gating charge moves all the way from the inside to the outside during opening. If it moved only part of the distance, the effective gating valence would be proportionally reduced. In Eqs. 17 and 18, at physiologic temperatures, the factor kT/q_{e} ∼ 25 mV, and thus if V is measured in millivolts, the previous equations can also be written more simply as \begin{equation}\tag{19}\alpha \left( V \right) = {\alpha _0}{e^{\delta zV/25}}\end{equation} \begin{equation}\tag{20}\beta \left( V \right) = {\beta _0}{e^{  \left( {1\,  \,\delta } \right)zV/25}}\end{equation} a form that for simplicity will be used in the models described in the present article. In case the temperature is considered an important variable, the equations can be adjusted accordingly. It can be seen that the gating of the voltagedependent 2state system depends on exactly 4 parameters: α _{0} and β _{0}, the rate constants at 0 mV; z, the gating valence; and δ, the relative electrical distance associated with opening.
The steadystate voltage dependence of the open probability turns out to be \begin{equation}\tag{21}{P_O}\left( V \right) = {{\alpha \left( V \right)} \over {\alpha \left( V \right) + \beta \left( V \right)}} = {1 \over {1 + {e^{  z\left( {V  {V_{{1 \over 2}}}} \right){q_e}/\left( {kT} \right)}}}} = {1 \over {1 + {e^{  z\left( {V  {V_{{1 \over 2}}}} \right)/25}}}}\end{equation} where V _{1/2}, the voltage of halfmaximal activation (P_{O}(V _{1/2} ) = 0.5) depends only on α _{0}/β _{0}. Importantly, P_{O}(V) is independent of δ. This equation is also called a Boltzmann distribution, because it fundamentally derives from the classical Boltzmann statistics describing systems in thermodynamic equilibrium.
Relaxation time constants are given by \begin{equation}\tag{22}\tau \left( V \right) = {1 \over {\alpha \left( V \right) + \beta \left( V \right)}} = {1 \over {{\alpha _0}{e^{\delta zV/25}} + {\beta _0}{e^{  \left( {1  \delta } \right)zV/25}}}}\end{equation}
Finally, the total predicted ion current of a channel model depends of the possibly voltagedependent singlechannel ion currents associated with each conducting state. For the 2state model, we will assume a linear current voltage relationship characterized by the singlechannel conductance γ and the reversal potential E _{rev} \begin{equation}\tag{23}i\left( V \right) = \gamma \left( {V  {E_{{\rm{rev}}}}} \right)\end{equation}
To simulate a classical K^{+} channel, we assume, for example, E _{rev} = −80 mV, similar to a neuronal potassium channel. Further properties of the 2state system will be discussed within the MarkovEditor tool in the next section.
IV. THE TIME DEPENDENCE: MACROSCOPIC AND MICROSCOPIC RELAXATIONS
Tutorial 2 describes how to insert the 2state model into MarkovEditor, and in the following, we will assume that this step is complete.
We will verify if MarkovEditor reliably reproduces the previously described general properties and explore the dependence on the various parameters of the model. Predictions of model calculations are shown in the upper panel on the left of MarkovEditor, and calculations are invoked by clicking on the Calculate button. To simulate the temporal development, either at the macroscopic or at the singlechannel level, we have to impose a voltage protocol, emulating the typical stimulation protocols used in electrophysiologic measurements. MarkovEditor offers 2 alternative stimulation protocols:

A simple protocol consisting of 3 stimulation segments is defined in the General Settings (see Tutorial 3 for a more exhaustive description of the General Settings).

More complex stimulation protocols can be defined by using the Pulse Generator File paradigm, explained in more detail in Tutorial 4.
Let us use the simple 3 stimulation segments protocol to simulate the response of the 2state model to a single voltage step. For this, using the menu item Settings>Edit general settings, we select NPulses = 1; sample time =0.1; VHold =100, t0=t_{Hold}=50; V1=20, t1=500; V2=80,t2=200 (the delta values are irrelevant if only a single pulse is simulated). The corresponding settings file 2States.set can be downloaded from https://github.com/mikpusch/MarkovEditor.git. This protocol is a standard pulse used to examine the behavior of classical voltagegated ion channels. The holding voltage of −100 mV is, on the one hand, close to the physiologic resting membrane potential of many excitable cells and, on the other hand, guarantees that the initial open probability is relatively small (for our 2state system). To invoke the simulation by MarkovEditor, we first select the radio button Current in the Time course tab and then click on the Calculate button to trigger the calculation shown in the graph on the top left (Fig 3A). The graph offers several display options and data can be easily exported, as described in Tutorial 5. We see that the initial negative holding current at −100 mV is very small, because the open probability is only 0.33%. Stepping the voltage to −20 mV leads to an exponential increase of the outward potassium current, which reaches almost steady state during the 500ms pulse. Upon returning the voltage to −100 mV, the initial tail current is relatively large, reflecting the initial sizable open probability of 65%, reached at the end of the preceding pulse to −20 mV, decaying back to the holding current, again with an exponential time course. Selecting the Prob radio button, displays the time course of the occupation probabilities of the closed state (in black) and the open state (in red; not depicted). By playing with the various options offered by the stimulation protocols, the macroscopic relaxations of the 2state model can be explored in an exhaustive manner.
V. STOCHASTIC SIMULATIONS OF SINGLE CHANNELS
We next explore the stochastic response of a single channel to the very same stimulation protocol described previously. To render the response more realistic, we artificially assign a noise level to both the open as well as to the closed state. This is done by doubleclicking or rightclicking on each state and assigning the noise level (sigma). In this manner, the current is associated with random noise with a standard deviation defined here. Note that before the setting of the noise, the user is asked to enter the label of the state and the current associated with it. This noise is typical for singlechannel recordings and reflects noise caused by a finite seal resistance, amplifier noise, or stray capacitance (11). For the closed state, we select sigma = 0.05 pA, whereas for the open state, we select higher noise level of 0.1 pA, reflecting the observation of open channel noise (15). Now, we select the radio button Current from the Simulation tab. Each time the Calculate button is clicked, the program simulates and shows the response of a single channel. As illustrated in Figure 3B, the stochastic behavior of the channel implies that the response is slightly different in the 3 simulations, even if the parameters are exactly the same. For a quantitative understanding of the behavior of the single channel, it might be insightful to select a large number of pulses in the General Settings dialog, export the data, and perform singlechannel analysis with a program of choice. This procedure is also helpful to assure the accuracy of any chosen analysis procedure.
VI. COMPARING DETERMINISTIC AND STOCHASTIC PREDICTIONS
Another instructive exercise is to compare the stochastic predictions of the model with the macroscopic, deterministic behavior. To this end, we can assign a large number of channels to be simulated in the GeneralSettings dialog, for example, No. Channels in stochastic simulations = 1000. Under these conditions, the noisy appearance of the stochastic prediction, compared with the deterministic one, is still visible, yet with 1,000 channels, the stochastic and the deterministic predictions become almost superimposable (Fig 3A,C). This kind of simulation can be conveniently used to verify the accuracy of programs used, for example, to perform nonstationary noise analysis.
VII. EXPLORING THE 2STATE MODEL WITH MARKOVEDITOR STEADYSTATE PROPERTIES
For the determination of steadystate properties, we have to specify the voltagerange over which to perform the calculations. These are inserted in the General Settings (see Tutorial 3 for a more exhaustive description of the General Settings). For example, we can impose the voltage range over which to calculate the steadystate properties to VLeft = 100 mV, VRight = 100 mV. As explained in the following, MarkovEditor can visualize the steadystate or timedependent occupation probability of up to 4 different states. The states to visualize are defined by the Probability Code variables of this dialog (Prob 1, Prob 2, Prob 3, Prob 4), which are set to states 0, 1, 2, and 3 by default but can be changed to any value, in particular for models with more than 4 states.
Now, selecting the I radio button under the Steady state tab and then clicking on the Calculate button will generate the graph illustrated in Figure 4A, which shows the net steadystate ion current predicted by the model. As usual, the graph offers several display options regarding the scaling of the axis and data can be easily exported, as described in Tutorial 5. The net steadystate ionic current is calculated as the sum \begin{equation}\tag{24}{I_{{\rm{net}}}} = \mathop \sum \limits_i {p_i}{c_i}\end{equation} where p_{i} is the (steadystate) occupation probability of state i and c_{i} is the current value of state i. Note that for the deterministic predictions of channel properties, such as the steadystate ion current discussed here, the predictions for a single channel are calculated. The number of channels defined in the settings is not taken into consideration. The number of channels is considered only for stochastic simulations (see the previous discussion). For the 2state potassium channel model here, the current of the fully activated channel follows a straight line for voltages more positive than +50 mV, as indicated by the dashed red line in Figure 4A. At more negative voltages, the current becomes progressively smaller, reflecting the reduced open probability. This is more evident in Figure 4B, which shows the voltagedependent steadystate occupation probability of the closed state (in black) and the open state (in red), respectively. This graph is selected by clicking on the P_{o} radio button. The Boltzmann distributions describing the occupation of the open and closed state, respectively, are nicely visible. Remaining in the Steady state tab, and clicking on the tau radio button shows, on a logarithmic scale, the voltage dependence of the relaxation time constant (Fig 4C), in full agreement with Eq. 22. The cycle button applies only to models with loops and will be discussed in the following in the context of models that aim to simulate transporters.
VIII. EXPLORING THE PARAMETER SPACE WITH VARY PARAMS
The predictions of any Markov model depend on the topology of the model (not discussed here) and on the rate constants between states. To explore the behavior of the model in a certain parameter space, MarkovEditor offers the possibility to vary, in a systematic manner, one or more parameters within a certain range of values. The parameters to vary and the range over which to vary them are defined in the dialog window Settings>Edit vary which params. As a first example we will vary the opening rate constant at 0 mV, i.e., α _{0}, in the range between 0.1 and 10, in 3 steps in a logarithmic manner, as detailed in Tutorial 6. Before initiating the exploration, we select the I radio button (i.e., to select Current to be calculated) in the Steady state tab and then click on the Vary params button. The resulting graph shows, superimposed with different grayscales, the I to V plots for the 3 different values of the opening rate constant (indicated explicitly in a separate text box in the plot window), as shown in Figure 4D. Clicking on the P_{o} radio button, results in the display of the corresponding occupation probabilities, nicely illustrating that the voltage of halfmaximal activation depends on the rate constants in an exponential manner (not depicted). Similarly instructive is the illustration of the dependence of the relaxation constant on the opening rate constant (not depicted).
Another important parameter to explore is the apparent gating valence encoded by in parameter a[2]. For example, setting the range between 1 and 4, in steps of 0.5, nicely illustrates the dependence of the steepness of the p _{open} (V) curve on the gating valence (not depicted).
IV. EXPLORING THE 2STATE MODEL WITH MARKOVEDITOR: THE POWER SPECTRUM
The noise associated with the random fluctuations of the channel between open and closed states contains useful information (16). This information can be extracted by measuring and expressing the noise in terms of the power spectrum, as explained in detail in (17, 18). In MarkovEditor, the power spectrum can be calculated by choosing the respective radio button. Explicit formulas for the power spectrum are shown in a text window. The spectrum will be calculated for different voltages, as defined by the first segment of the simple pulse protocol in the general settings dialog. For example, setting

V /cl = −60

delta V / cl = 0

NPulses = 5
spectra will be calculated for −60, −40, −20, 0, and 20 mV. The 2state model shows a typical single Lorentzian dependence on the frequency (not depicted). The cutoff frequency of the spectrum corresponds to the inverse time constant of the macroscopic relaxation. Note that the spectra calculated by MarkovEditor do not take into account the sigma values associated with the singlechannel currents.
X. SINGLECHANNEL BEHAVIOR IN MORE DETAIL: THE DWELLTIME DISTRIBUTIONS AND THE SIMPLEST LIGAND ACTIVATED CHANNEL
A standard analysis of singlechannel recordings consists in the construction of the distributions of the dwell times in the various conductance states. The dwelltime distributions are then fitted with the sum of exponential functions. The number of exponential components needed to satisfactorily describe the distribution provides the number of states underlying the respective conductance state. For example, a single exponential distribution is indicative of a single state, a double exponential distribution indicates 2 states with the same conductance level (10). To illustrate this behavior, we will create a 3state model, with 2 closed states and 1 open state. To render the model even more related to real life, we will interpret it in terms of a ligand activated channel, as illustrated in the following scheme: \begin{equation}\tag{25}U\matrix{ {\mathop \to \limits^{\lambda \left[ L \right]} } \cr {\mathop \leftarrow \limits_\mu } \cr } B\matrix{ {\mathop \to \limits^\alpha } \cr {\mathop \leftarrow \limits_\beta } \cr } O\end{equation}
Ligand L binds to the unliganded state U with rate constant λ[L], where [L] is the ligand concentration and dissociates from bound state B with rate constant μ. The bound state opens with rate constant α and the open state O closes with rate constant β. The model is entered in MarkovEditor as explained in Tutorial 7. To become acquainted with the model, we will first explore its macroscopic steadystate and kinetic properties. The ligand concentration [L] is encoded in the independent variable c, playing an equivalent role as the voltage in voltagegated channels. Having defined the range of [L] in the general settings, as described in Tutorial 7, we can now calculate the doseresponse behavior by using the Steady State tab and responses to concentrations steps by using the Time Course tab. Note that the maximal open probability reaches only 50% at saturating ligand concentration.
We next simulate singlechannel behavior in steady state at a fixed ligand concentration, for example a low concentration of 0.1 mM. To this end, we set all concentrations of the simple stimulation protocol (see Tutorial 3) to the same value 0.1, select 1 pulse, and a sample time of 1 ms. To achieve a long simulation time of 100,000 ms, we set the duration of segment 1 to 100,000 (this field is denoted t1 (ms)) in the General Settings dialog. (The corresponding settings file LigandGatedDwellTimeDistribution.set can be found on https://github.com/mikpusch/MarkovEditor.git). Now selecting the Current radio button in the Simulation tab and clicking on Calculate will produce a simulated singlechannel trace, which should show typical bursts of openings, separated by long shut intervals, as illustrated in Figure 5A. Indeed, such bursts are encountered frequently in real singlechannel recordings (11). We do not attempt to define precisely the term burst here but will use this example to illustrate the concept of dwelltime distributions.
Selecting the tab Dwell Times and clicking on Calculate will invoke the calculation of theoretic steadystate dwelltime distributions of the closed states and of the open states, respectively. These will be calculated at different concentrations or voltages, as defined by the first segment of the simple pulse protocol in the general settings dialog. For example, setting

V /cl = 0.1

delta V / cl = 0.9

NPulses = 2
dwelltime histograms will be calculated for c = 0.1 and for c = 1, as illustrated in Figure 5B. (The corresponding settings file LigandGatedDwellTimeDistribution.set can be found on https://github.com/mikpusch/MarkovEditor.git.) The cumulative distributions, P _{cum} (t), provide the probability that a dwell time is longer than time t. By definition, P _{cum}(0) = 1 and P _{cum} (t) decreases monotonically for longer times. The cumulative dwelltime distribution can be used to calculate the probability that a dwell time lies in a certain time range [t, t+ Δt] by the formula Prob(duration of event in range [t, t+ Δt])= P_{cum}(t)P_{cum}(t+Δt) , which is the basis for fitting dwelltime histograms, as shown in Figure 1C. In MarkovEditor, the text window in the top middle shows the analytic expressions for the dwelltime distributions. Note that the dwelltime distribution of the open state is a single exponential (with time constant 1/β), whereas the closed time distributions are double exponential, with time constants (and coefficients) that depend on the remaining rate constants and the ligand concentration. Note that the theoretic dwelltime distributions assume infinite time resolution, i.e., that all, even the shortest events are detected. In reality, often the missed events problem is far from being negligible. See (19–22) for more details on singlechannel analysis.
XI. A COMPLEX MODEL OF A VOLTAGEGATED SODIUM CHANNEL: IONIC AND GATING CURRENTS
To show the capability of MarkovEditor to deal with realistic models, we will illustrate the behavior of a model of voltagegated Na^{+} channels described by Patlak (23). At the same time, this section will serve to introduce the possibility to enter, share, and edit models by using a textbased interface. Gating of voltagedependent Na^{+} channels involves (at least), 2 processes: activation and inactivation, leading to transient activation of Na^{+} currents upon voltage steps. In the classical Hodgkin–Huxley model, full activity is achieved by the simultaneous activation of 3 m gates, whereas inactivation is modeled by a single h gate, leading to a rather complicated model with several closed and inactivated states. For a more detailed introduction the reader is referred to (23, 24). In particular, the model we will consider is Model 7 of (23), which is inserted into MarkovEditor, as illustrated in Tutorial 8. This should result in a model display, as shown in Figure 6A (do not forget to save the model in a file to store the work done). Before going into more detail, note that parameter w[0] of the model simply serves to encode the prefactor, kT/h, used by (23) to scale all rate constants (h is the Planck constant, h = 6.63 × 10^{−34} Js).
This model harbors a novel feature not discussed so far: functions. Functions are not strictly necessary, but they make life easier in certain cases. Functions conceptually differ from variables in the following way: variables are evaluated exactly once (and in order from w[0], w[1], and w[2]) when transition rates and state currents need to be calculated. On the other hand, functions are evaluated only when explicitly called within variable or state current expressions or from other functions and can be called multiple times (note that nonterminating recursive function calls will result in program block).
The function func[0] in the model has been introduced to implement a restriction, proposed by Patlak (23), on the maximum value of rate constants to avoid unphysically large rate constants at extreme voltages caused by the exponential voltage dependence of the expressions for the various rate constants (w[1] through w[8]). Slightly differently from the original Hodgkin and Huxley equations, Patlak used the expression \begin{equation}\tag{26}{1 \over {{\rm{effective\ rate}}}} = {1 \over {{\rm{calculated\ rate}}}} + {1 \over {{\rm{maximum\ rate}}}}\end{equation} that is implemented in \begin{equation}\tag{27}{\rm{func}}\left[ 0 \right]\left( x \right) = {{x*a\left[ {13} \right]} \over {x + a\left[ {13} \right]}}\end{equation} which returns the effective rate, given the calculated rate x, and the maximum rate as set in parameter a[13] (set to 20,000 s^{−1} in the model). As can be seen in the model definition (Fig 6A and Tutorial 8), func[0] is applied to every single rate constant connecting the various states. This example highlights the advantage of using a reusable function in simplifying complex expressions, and it will be used in the following for the prediction of gating currents.
Similar to the 2state model described in the first paragraphs, the open channel current is given by \begin{equation}\tag{28}i\left( V \right) = 0.01*\left( {V  50} \right)\end{equation} to emulate a singlechannel conductance of 0.01 nS (10 pS) and a positive reversal potential, leading to excitatory inward currents at physiologic voltages. Predictions of macroscopic and single currents upon typical voltage protocols are shown in Figure 6B,C.
In addition to the ionic currents, MarkovEditor provides the possibility to simulate socalled gating currents without any further modification of the model. Gating currents (25) arise from the movement of charges within the membrane upon changes of the membrane potential. They are of capacitive nature (i.e., transient). For voltagegated cation channels, the major part of the gating currents is mediated by the movement of the charged S4 segments (25), which leads to channel opening. In fact, the voltage dependence of the rate constants arises directly from these charge movements. Each time a voltage sensor moves outward, a tiny shotlike outward current event occurs, whose charge (i.e., time integral over the current) is equal to the effective charge movement. When the voltage sensor moves back inward, a similar inward “shot” current event is generated. On the other hand, these charge movements are the basis of the exponential voltage dependence of the rate constants (see Fig 2C). With these assumptions, the average gating current can be calculated using the rather complicated expression \begin{equation}\tag{29}{I_{{\rm{gating}}}} = {q_e}\mathop \sum \limits_i {p_i}\left( {\mathop \sum \limits_{j \ne i} {\rm{rate}}\;{\rm{constant}}\left( {i \to j} \right){z_{ij}}} \right)\end{equation} where, as in Eqs. 16–18, z_{ij} is the gating valence conferring the voltage dependence to the transitions between stats i and j (z_{ij} = −z_{ji} ), and p_{i} is the probability to reside in state i. The concepts underlying the considerations leading to Eq. 29 are beautifully described in (25). Fortunately, MarkovEditor determines in a completely automatic manner the gating valences and all other parameters needed (see section XIX), allowing a straightforward prediction of gating currents.
Just 2 simple steps are needed to instruct MarkovEditor to calculate gating currents (instead of the standard ionic currents):
First, we have to check the checkbox Use transporter/gating current and uncheck the checkbox Use channel current.
Second, we have to insert the string auto as the Transporter/gating current function meaning that gating currents are automatically determined by using Eq. 29.
Alternatively, we could insert manually any expression for the Transporter/gating current function. This alternative mode of inserting the gating current function can be useful to explore any specific property of a model and is explained in Tutorial 9.
Macroscopic gating currents for the Patlak Na^{+} channel model are shown in Figure 6D, illustrating typical features, such as a rising phase at certain voltages. This feature differs from the predictions of the original Hodgkin–Huxley model, for which gating currents should be a decaying exponential function (23). Another feature correctly reproduced by the model of Patlak is charge immobilization, which is visible in the asymmetry between on and offgating currents (25). To better understand the nature of gating currents, it is also instructive to simulate them at the stochastic level, as illustrated in Figure 6E. Each time the channel undergoes a transition, a spikelike gating current, which cannot be resolved in normal patchclamp measurements, is generated. Analysis of the nonstationary noise associate with gating currents has been used by Conti and Stühmer (26) to obtain direct experimental evidence for the quantal characteristic of gating currents.
XII. VARIATION OF A PARAMETER: INSIGHT INTO POTENTIAL DISEASE MECHANISMS
Voltagegated sodium channels play a role in most excitable cells (neurons, skeletal, muscle, heart, and endocrine cells) and defects in the respective encoding genes lead to a plethora of human genetic diseases (27, 28). A rather frequent phenotype of disease causing mutations is the appearance of socalled “persistent” currents, or more generally, defects that render the inactivation process less efficient; see (29–31). For example, the mutation T1313M in the muscle Nav1.4 channel causes paramyotonia congenita and is associated with significant persistent currents, shifted voltage dependence of inactivation, and faster recovery from inactivation (29). MarkovEditor allows us to develop hypotheses about the molecular mechanisms underlying such a functional defect, as illustrated in Figure 7. We concentrate on the single parameter α_{h} _{0}, which is the rate constant important for recovery from inactivation. The value of α_{h} _{0} is determined by parameter a[9] and calculated in variable w[7] in the model (see Tutorial 8). For wildtype (WT), a[9] = 26.5, which via variable w[7] = exp(w[0] + a[9]  a[11]*(1a[12])*v/25) results in α_{h} _{0} (i.e., at 0 mV) ∼ 19.3 s^{−1}. To increase α_{h} _{0} efold, i.e., by a factor of 2.71, we set a[9] = 25.5, resulting in α_{h} _{0} ∼ 52.6 s^{−1}. This simple maneuver leads to a number of changes. Figure 7A,B show the response of WT channels and mutated channels to a typical voltageprotocol used to measure steadystate inactivation. Clearly, the mutated channels exhibit much larger persistent currents. After the long pulses to various voltages, channel availability is measured by a final test pulse to 0 mV, and the resulting (absolute) peak values are plotted in Figure 7C as a function of the conditioning voltage. The solid lines are fits with a Boltzmann distribution. The mutant data are slightly shifted to more positive voltages. Finally, Figure 7D and E illustrate the response to a typical protocol used to measure the speed of recovery from inactivation. After a 70ms inactivating prepulse, the membrane voltage is kept at −90 mV for increasing time, and channel availability is measured by a test pulse to 0 mV. WT channels recover in a monoexponential manner (gray line in Fig 7D) with a time constant that is 3 times slower than mutant channels (red line in Fig 7E). This is just a simple example of how MarkovEditor can be used to glean mechanistic inside into the effect of diseasecausing mutations.
XIII. A SIMPLE ELECTROGENIC UNIPORTER
Even though initially designed to model ion channels, MarkovEditor is well capable of simulating even complex transporters, both at the steadystate level, and in terms of transient currents, which is relevant for electrogenic transporters. This will be illustrated here using a simple electrogenic uniporter, described by 4 states, as illustrated in Figure 8. The uniporter can expose its binding site either to the outside (upper states) or to the cytosol (lower states). The conformational change between these configurations is assumed to be intrinsically voltage dependent (indicated by the movement of the “green” negative charge in Fig 8), and translocation of the positively charged substrate confers an additional voltage dependence to the loaded transporter. Binding and release of substrate may also be voltage dependent. For example, this is the case for Na^{+} binding in the Na–K–ATPase (32). This specific transporter model incorporates several features highlighted and investigated by researchers (33–35). The model can be inserted into MarkovEditor using the textbased method, as detailed in Tutorial 10. Alternatively, the model file Uniporter.mod can be downloaded from https://github.com/mikpusch/MarkovEditor.git. Regarding the equations and symbols depicted in Figure 8, the parameters of the model are used with the following conventions:

a[0]:
δ _{4}, effective valence of the charge associated with the conformational change from outside to inside, independent of substrate binding. The parameter combines both the actual charge as well as the electrical distance associated with the charge movement into a single number. In effect, for a complex protein, the simplification consists in replacing by a single effective valence the movement of diverse charges and dipole moments in the apoprotein during its structural transition. The value of a[0] is negative to emulate a negative intrinsic charge!

a[1]:
δ _{1}, electrical distance of the binding site exposed to the outside.

a[2]:
δ _{3}, electrical distance associated with moving of substrate from outer open state to inner open state.

1a[1]a[2]:
δ _{2}, electrical distance of the binding site exposed to the inside.

a[4]:
α _{0}, rate constant (at 0 voltage) of conformational change from outside exposing to inside exposing binding site of empty transporter.

a[6]:
α _{1}, rate constant (at 0 voltage) of conformational change from outside exposing to inside exposing binding site of loaded transporter.

a[7]:
β _{1}, rate constant (at 0 voltage) of conformational change from inside exposing to outside exposing binding site of loaded transporter.

a[8]:
k _{off} ^{int} = k _{off} ^{ext}, unbinding rate constant of substrate (at 0 voltage) of substrate to inside or outside, assumed to be equal.

a[9]:
k _{on} ^{int} = k _{on} ^{ext}, association rate constant of substrate (at 0 voltage) of substrate from inside or outside, assumed to be equal.

a[32]:
c _{ext}, extracellular substrate concentration.

a[33]:
c _{int}, intracellular substrate concentration.
Note that for simplicity (to reduce the number of parameters), the affinity of the outwardly oriented binding site is assumed to be identical to that of the inwardly oriented binding site.
An extremely important aspect regarding models of transporters, or more generally models involving loops or cycles, is the second law of thermodynamics and the impossibility to construct a perpetuum mobile. For example, for the uniporter model the following sequence of state occupancies

Out 0 → Out 1 → In 1 → In 0 → Out 0
would represent a clockwise cycle of transport corresponding to an inward transport of a substrate molecule. However, to achieve a completed inward transport cycle, the transporter may go back and forth between states several times. Therefore, also the following sequence of state occupancies would be a clockwise cycle associated with the inward transport of a substrate molecule:

Out 0 → Out 1 → In 1 → Out 1 → In 1 → In 0 → In 1 → In 0 → Out 0.
By analogy, out of many possible cycles, the inverted sequence of the previous cycle

Out 0 → In 0 → In 1 → In 0 → In 1 → Out 1 → In 1 → Out 1 → Out 0
represents an anticlockwise cycle, corresponding to the outward transport of a substrate molecule. To be physically realistic, Markov models have to obey the second law of thermodynamics. This implies that in conditions of thermodynamic equilibrium, no net cycling can occur for any cycle of the Markov scheme. This means that, on average, the number of cycles completed in the clockwise direction has to be identical to the number of cycles completed in anticlockwise direction (36). Otherwise, the transporter would be able to accumulate substrate on one side of the membrane without external energy input. This principle is also called microscopic reversibility and applies only to equilibrium conditions. In case of the electrogenic uniporter discussed here, equilibrium conditions are zero transmembrane potential and equal substrate concentrations on the inside and on the outside. Fortunately, this constraint can be easily incorporated into the model because it is equivalent to the requirement that, for each cycle, \begin{equation}\tag{30}\mathop \prod \limits_{{\rm{clockwise}}} {\rm{rate}}\;{\rm{constants}} = \mathop \prod \limits_{{\rm{anticlockwise}}} {\rm{rate}}\;{\rm{constants}}\end{equation} i.e., that the product of rate constants in the clockwise direction of each cycle is equal to the product calculated in the anticlockwise direction (36). In equilibrium conditions, i.e., V=0, c_{ext}=c_{int} , for the uniporter model, the previously mentioned condition translates into \begin{equation}\tag{31}k_{{\rm{on}}}^{{\rm{ext}}}{\alpha _1}k_{{\rm{off}}}^{{\rm{int}}}{\beta _0} = k_{{\rm{on}}}^{{\rm{int}}}{\beta _1}k_{{\rm{off}}}^{{\rm{ext}}}{\alpha _0}\end{equation}
Because we assumed the binding or unbinding properties to be equal for both inward and outwardfacing configurations of the transporter, the constraint can be simplified and reformulated as \begin{equation}\tag{32}{\beta _0} = {{{\alpha _0}{\beta _1}} \over {{\alpha _1}}}\end{equation}
This has been incorporated in the model in that variable w[1] is calculated as \begin{equation}\tag{33}{\mathtt{w \left[ 1 \right] = a \left[ 4 \right] * a \left[ 7 \right]/ a \left[ 6 \right]}}\end{equation} which is then used to calculate β _{0}.
After these lengthy, but important considerations we are almost ready for the predictions of the uniporter model. To calculate the current generated by the model, we have to check the checkbox Use transporter/gating current and uncheck the checkbox Use channel current, and we have to insert a valid Transporter/gating current function. As for the gating currents of the voltagegated sodium channel model, the easiest choice for this is auto, which derives the charge translocation steps from the voltage dependence of the rate constants.
Importantly, it is possible to insert any syntactically valid function as the Transporter/gating current function, allowing the analysis of every aspect of the model.
For example, for a singlecycle model as the uniporter model, the net cycling frequency is proportional to the net transport activity, which can be expressed as \begin{equation}\tag{34}{\mathtt{p \left[ 2 \right] * w \left[ 5 \right]  p \left[ 0 \right] *w \left[ 7 \right]}}\end{equation}
Here, p[2] is the (timedependent) occupation probability of state 2, and w[5] the rate constant of transitions from state 2 (outside oriented, substrate bound) to state 0 (outside oriented, no substrate bound, “apo”). Thus, the product p[2]*w[5] is the effective rate of transitions from state 2 to state 0. Similarly, the product p[0]*w[7] is the effective rate of transitions from state 0 to state 2. For any cyclic scheme, the difference of such rates of neighboring states provides the net cycling frequency in steadystate conditions (36). Thus, Eq. 34 can be used alternatively as the Transporter/gating current function for the purpose of steadystate calculations. However, with this equation, not all the transitions contributing to the transient currents are correctly taken into account resulting in errors in the calculation of transient (or presteadystate) currents, as well as the power spectrum. Therefore, for the electrogenic uniporter discussed here, we will stick to the choice of auto for the Transporter/gating current function.
In addition to microscopic reversibility, care has to be taken in assigning the correct voltage dependence to each rate constant to maintain a correct treatment of net inward versus outward charge movement. For the electrogenic uniporter model, the formal correctness of the equations can be checked by assuring that the calculated reversal potential corresponds to the Nernst potential (6). At the Nernst potential, outwardly directed transport cycles occur at the same (average) frequency of inwardly directed transport cycles. An uphill transport of the substrate against the electrochemical equilibrium in the absence of an external energy input would violate the thermodynamic laws. The Nernst potential is given by \begin{equation}\tag{35}{E_{{\rm{rev}}}} = {{kT} \over {{q_e}}}\ln\left( {{{{c_{{\rm{ext}}}}} \over {{c_{{\rm{int}}}}}}} \right) \approx 25\ {\rm mV}\ln\left( {{{{c_{{\rm{ext}}}}} \over {{c_{{\rm{int}}}}}}} \right){\rm }\end{equation} (at room temperature kT/ \({q_e}\) ∼ 25 mV). We will perform this test in a first exercise. We assign

a[32] = 10 = c _{ext} and a[33] = 1 = c _{int}.
Inserting these 2 concentrations into Eq. 35 yields a theoretic reversal potential of E _{rev} = 25 mV×ln(10) = 57.6 mV. To simulate the steadystate IV relationship, we check the I radio button in the Steady State tab and click on the Calculate button, which should result in the display of the steadystate IV as shown in Figure 9A. Exporting the data to a spreadsheet (via rightclicking on the graph) allows to verify that the simulated data closely follow the predicted reversal potential.
The Vary params feature allows us to investigate the dependence of transport on intracellular and extracellular substrate concentration, as well as on all other parameters. As an example, Figure 9B illustrates the effect of varying the intrinsic charge associated to transport function (δ _{4} in Fig 8).
XIV. PRESTEADYSTATE CURRENTS OF THE UNIPORTER
Upon abrupt changes of the membrane potential, the initial relaxation of the transporter toward a new steadystate condition, as exemplified in Eq. 11 for the 2state model, results in transient or presteadystate currents, before reaching the steady transport level. In this case, we use auto as the Transporter/gating current function and the current relaxations faithfully reflect the predicted transient (and steadystate) currents of the model, as all transitions are taken into account. These transient currents provide important information on the transport mechanism, but they are often difficult to measure because they are partially masked by the capacitive currents associated with the charging of the cell membrane capacitance. In specific cases, this problem can be partially circumvented by comparing the transient currents in the presence and in the absence of the substrate (37). For the 4state uniporter investigated here, typical ideal transient currents are illustrated in Figure 9C.
XV. THE TRANSPORTER SPECTRUM
In any condition, even at thermodynamic equilibrium, the electrogenic stochastic transitions of the transporter, which might be either substrate binding or unbinding events, or conformational changes, are associated with tiny spikes of the electric current, as already discussed in the framework of the gating currents of the sodium channel. For a given Markov model, the power spectrum of the noise generated by these transitions can be predicted in an explicit manner (34, 35). In principle, the power spectrum could provide important information on the underlying transport mechanism, but, unfortunately, experimentally the power spectrum is difficult to measure in real life. In MarkovEditor, the spectrum will be calculated for different voltages, as defined by the first segment of the simple pulse protocol in the general settings dialog. For example, setting

V /c1 = −100

delta V / c1 = 100

NPulses = 3
spectra will be calculated for −100, 0, and 100 mV. MarkovEditor visualizes the power spectra at these voltages, both graphically as well as in analytic formulas. The typical spectrum of a transporter is qualitatively different from that of a channel because of the shot noise properties of the electrical events associated with transporter activity. The examples for the uniporter, shown in Figure 9D, are calculated at −100, 0, and 100 mV with c _{ext} = 10 mM, c _{int} = 1 mM. The spectra rise from a lowfrequency limit to a highfrequency limit with 3 separate Lorentzian components. The correctness of the theoretic spectra can be verified by stochastic simulation of a large number of transporters and calculation of the power spectrum on the exported data.
XVI. CYCLING FREQUENCY OF THE UNIPORTER
For transporters, and more general, for all Markov models that contain closed loops, the frequency of clockwise and anticlockwise cycling is an interesting property. For example, as noted previously, in thermodynamic equilibrium, these 2 cycling frequencies have to be equal. For a transporter with a single cycle, as the 4state uniporter, the difference in clockwise and anticlockwise cycling frequency determines the effective transport rate. Using the method of Hill (36), MarkovEditor is able to calculate the clockwise and anticlockwise cycling frequencies for each cycle. The cycle to be calculated has to be chosen by clicking on Select cycles. For the uniporter model, only a single cycle is present. After the selection, checking the radio button cycle and clicking on Calculate, will produce the clockwise (in red) and anticlockwise (in black) cycling frequencies (in units of s^{−1}; Fig 10A). At the reversal potential, these 2 frequencies are equal. Note that for models with a large number of cycles, the algorithm used is not very efficient.
XVII. SINGLE TRANSPORTER SIMULATIONS
At present, the electrical current associated with single transporters is beyond experimental resolution. However, singlemolecule fluorescence techniques have been used to follow conformational changes of certain, rather slow bacterial transporters (38). The simulation of single transporter activity can thus provide a guide to interpret singlemolecule measurements. MarkovEditor can, on one hand, simulate the single transporter electrical activity, as illustrated in the section on gating currents of the voltagedependent sodium channel. This feature will be used in XVIII section to validate the calculation of the power spectrum of transporter generated currents.
On the other hand, the following trick can be used to visualize the stochastic behavior of a single transporter molecule. To follow the transitions of a single transporter in time, we artificially assign a distinct current value to each state. In this way, the simulated current trace of a single transporter of the transporter will reflect the transitions between the various states. In the example illustrated in Figure 9D, we chose the following values:

3 for state Out 0

2 for state Out 1

1 for state In 0

0 for state In 1
Furthermore, we uncheck the checkbox Use transporter/gating current and check the checkbox Use channel current, to visualize the fake specificstate associated ion currents. Then, assigning asymmetric ion conditions and simulating (e.g., for 100 ms), Figure 10B clearly illustrates the net clockwise cycling. The trace shows a clear asymmetry regarding the direction of time. In equilibrium conditions (e.g., at the Nernst potential), no statistical method should be able to distinguish the timereversed signal from the ordinary signal. In this context, an instructive example is the singlechannel analysis of Richard and Miller of the Torpedo chloride channel, which revealed that channel gating is not at thermodynamic equilibrium (39).
XVIII. VISUAL SIMULATION OF A SINGLE TRANSPORTER
An alternative way to explore the behavior of a single channel or transporter is provided by the visual simulation. With this feature, the timedependent simulated stochastic behavior of a single channel, calculated according to the simple protocol defined in the General Settings, is visualized in a timedependent manner by highlighting the actually occupied state in red and the previously occupied state in blue (see Fig 10C). Furthermore, the previously occupied state and the currently populated state are connected by a colored arrow. The frame rate at which the simulation is reproduced is defined by the time step in visual simulation in the General Settings. In other words, each time step of the simulation (defined by the sample time) is going to be visualized on screen for the duration defined by the time step in visual simulation (the settings file UniporterVisualSimulation.set can be downloaded from https://github.com/mikpusch/MarkovEditor.git) In parallel, the graph in the upper pane shows the current time point of the simulation protocol. The currently active protocol value is shown as a number, and a small colored arrow indicates the progress through the protocol. To invoke the visual simulation, select Visual in the Simulation tab and click Calculate. To interrupt an ongoing visual simulation, simply click on the Interrupt button or click again on Calculate. It may happen that the colored arrow is connecting states for which no direct transition is actually permitted. This behavior is possible if, during a single sample period, more than one transition occurs. In fact, as described in section XIX, such missed events are realistically modeled in MarkovEditor. The visual simulation is not limited to transporter models but is applicable to every type of model.
XIX. METHODS
A. Method of deterministic numeric simulation
The Markov model is defined by a number of states N, connected by voltagedependent rate constants r_{ij} for transitions from state i to state j. For deterministic simulations, the timedependent occupation probabilities p_{i}(t) are governed by the differential equations \begin{equation}\tag{36}{{d{p_i}} \over {dt}}\left( t \right) = \mathop \sum \limits_{i \ne j} {p_j}{r_{ji}}  {p_i}\mathop \sum \limits_{i \ne j} {r_{ij}} = \mathop \sum \limits_j {p_j}{q_{ji}}{\rm }\end{equation} defining the Q matrix given by the components q_{ij} . The program can handle up to 100 states. Stationary occupation probabilities are denoted by π _{i} for state i and are calculated by standard matrix inversion. For temporal analysis, the (possibly complex) eigenvalues and eigenvectors of the Q matrix in MarkovEditor are determined by using the ALGLIB C library (http://www.alglib.net). For nonconstant segments (i.e., ramp, sine, wave; see Tutorial 6), numeric solution is performed by using an eighthorder Dormand–Prince integration algorithm (40). This may lead to slow execution in case of complex models.
For ion channel models, each state i is associated with a voltagedependent current c_{i} . In the tool described here, an arbitrary voltagedependent function can be assigned for the current of each state. A common simplified assumption is that the open channel current depends in a linear manner on the membrane voltage. For example, for a 50pS channel with an assumed reversal potential of −30 mV, a suitable function to describe the open channel current is given by 0.05 × (v + 30) with the implicit convention to measure voltage in millivolts and current in picoamperes.
For closed states, the current function should be 0. In addition to the current for each state, the simulation program also provides the opportunity to assign a noise level for each state (given as the standard deviation in picoamperes). This is only used for the stochastic simulation of a discrete number of channels (see the following).
Having assigned the microscopic current c_{i} for each state i, the predicted channel current, I _{chan}, in the deterministic mode, is given by \begin{equation}\tag{37}{I_{{\rm{chan}}}}\left( t \right) = \mathop \sum \limits_i {p_i}\left( t \right){c_i}\end{equation}
B. Transporter and gating currents
In addition to the open channel current, c_{i} , transitions between states may be associated with a charge movement reflecting for example gating charge movement in voltagegated channels or, more general, any electrogenic step in a transporter or channel. On a microscopic scale, these electrogenic steps correspond to a current spike, idealized by a delta function, whose time integral is given by the charge moved in the step. On a macroscopic, deterministic scale, the electrogenic steps produce an average, transient current I _{trans}, given by \begin{equation}\tag{38}{I_{{\rm{trans}}}}\left( t \right) = \mathop \sum \limits_{i,j} {p_i}\left( t \right){r_{ij}}{Q_{ij}}{\rm }\end{equation} where Q_{ij} is the charge moved upon the transition from state i to state j.
Within MarkovEditor, there are 2 different ways to calculate I _{trans}:

In the most general manner, it is possible to explicitly define a function, called the Transporter/gating current function, used to calculate I _{trans}, as illustrated for an example in the Tutorial 9.

In most models of channels and transporters, transition rates depend exponentially on the applied voltage, with the translocated charge, Q _{ij} , determining the voltage dependence. For example, the opening and closing rate constants, α (V) and β (V), respectively, might be given by \begin{equation}\tag{39}\alpha \left( V \right) = {\alpha _0}{e^{{{{z_\alpha }VF} \over {RT}}}}\end{equation} and \begin{equation}\tag{40}\beta \left( V \right) = {\beta _0}{e^{{{{z_\beta }VF} \over {RT}}}}\end{equation} where α _{0} and β _{0} are the respective rate constants at 0 mV, z_{α} and z_{β} the respective gating valences, and F, R, T the Faraday constant, the gas constant, and the absolute temperature, respectively, with kT/q_{e} ∼ 25 mV at room temperature. For such a transition, the value of the translocated charge for the opening transition is (z_{α} − z_{β} )×q _{e}, where q_{e} is the absolute charge of an electron. The charge for the closing transition is the negative value of that of the opening transition, that is (z_{β} − z_{α} )×q_{e} . More generally, the translocated charge of transition (i,j) at a given voltage can be determined by the logarithmic derivative with respect to voltage of the rate constants involved in the transition: \begin{equation}\tag{41}{Q_{ij}} = \left( {{d\ln(r_{ij})} \over {dV}}  {{d{\rm{ln}}\left( {{r_{ji}}} \right)} \over {dV}} \right)kT\end{equation} where k is Boltzmann's constant and T is assumed to be 300 K. This procedure for determining the translocated charges is selected in the simulation program by choosing auto as the Transporter/gating current function in MarkovEditor (or by leaving this field blank). With this setting, MarkovEditor uses the logarithmic derivative of the rate constants to determine the translocated charge (Eq. 41), and Eq. 38 to calculate the transporter and gating currents associated with the model. This choice is the simplest and, in most cases, an adequate mode of selection.
C. Method of stochastic simulation
So far, we have described the use of deterministic simulations of the properties of a finite number of ion channels or transporters. However, one could also use a stochastic approach (analyze the behavior of the ensemble starting from the properties of a single channel or transporter), under the assumption that they behave independently of each other. In this case, because each channel or transporter is modeled independently, the time needed for simulation is proportional to the number of channels and becomes prohibitive for a very large number of channels.
This mode of operation is useful, for example, to explore the variability of possible experimental results, if the underlying data would be described by a given Markov model. This mode also allows to test predictions of stationary and nonstationary noise analysis.
In the stochastic simulation mode of channels it is possible to include a noise level for each channel state. The noise is specified as the standard deviation of a Gaussian white noise level added on top of the current level of a channel state. This can be used for example to create realistically looking channel events, as we saw in Figure 4, or to test the influence of open channel noise on noise analysis or other types of analysis. It is also useful to test the capabilities of singlechannel analysis software, in particular, regarding missed events detection. The stochastic simulation of a single transporter or channel is exact in the sense that it is independent from the sampling time selected in the sampling procedure.
At the beginning of each sweep, the holding potential determines a starting state, using a random number chosen between 0 and 1, and the stationary occupation probabilities π _{i}. For example, for a 2state model with π _{0}= 0.3, π _{1}= 0.7, a random number <0.3 would lead to the selection of state 0 as the starting state, whereas a larger random number would select state 1. Random numbers are calculated by using the Mersenne Twister pseudorandom number generator (http://www.math.sci.hiroshimau.ac.jp/∼mmat/MT/MT2002/CODES/mt19937ar.c). Once a state i has been specified, the mean sojourn time in this state is given by the inverse of the sum of all rate constants leaving that state (11): \begin{equation}\tag{42}{\tau _i} = {1 \over {\mathop \sum \nolimits_j^{} {r_{ij}}}}\end{equation}
Using a random number r in the interval ]0…1], a stochastic prediction of the time spent in this specific state is given by \begin{equation}\tag{43}T =  \ln \left( r \right)*{\tau _i}\end{equation}
If the command voltage changes to a new value at any sampling instance before reaching time T, the rate constants are recalculated and a new dwell time is computed. The approach exploits that by virtue of the digitalization and discrete sampling, the command voltage changes stepwise. The procedure to calculate a new dwell time, independent of the time already spent in a given state, is valid because of the memoryless nature of a Markov process. At the end of the dwell time, the state to which the system will jump is determined with a novel random number and using the rate constants departing from the current state as relative weights in the decision. The timing of the jump is independent from the sampling time, and it may indeed happen that the system is changing state more than once during a given sampling period, leading effectively, and in a realistic manner, to missed events in the simulation.
To simulate the transporter or gating current spikes, the charge associated with the transitions is summed during each sample period providing Q _{net}, the net charge translocated during a sampling period. Then, the current signal Q _{net}/Δt, where Δt is the sampling interval, is added to the overall current sampling output.
XX. TUTORIAL 1: INSTALLING MARKOVEDITOR ON A WINDOWS COMPUTER
MarkovEditor is specific for Windows (Microsoft Corporation, Redmond, WA) and runs on Windows XP (Microsoft) and higher (including Windows 10, Microsoft). The code is written in Microsoft Visual C++, and although it is fully compatible with VisualStudio 2019 (Microsoft), the current build is done with VisualStudio 2005, allowing execution on Windows XP, which is still in use in many less developed countries. The source code is available on request. The program should run on Windows emulators on Mac (Apple Inc., Cupertino, CA) and Linux (Linux Foundation, San Francisco, CA).
You can download the MarkovEditor master zip file from https://github.com/mikpusch/MarkovEditor.git. Save the program file MarkovEditor.exe in a directory of choice and create shortcuts on the desktop. In addition, the master zip file contains the following model files containing the models illustrated in this article:

2States.mod

LigandGated.mod

Patlak_NaChannel.mod

Patlak_NaChannel_Mut.mod

Uniporter.mod
These model files can be loaded via the Menu item File>Open.
Further, the following setting files are included

2States.set

LigandGated.set

LigandGatedLongSimulation.set

LigandGatedDwellTimeDistribution.set

NaChannelIV.set

UniporterVaryParams.set

UniporterVisualSimulation.set
Finally, the pulse generator file (pgf), Pulses.pgf, is included. It contains several sample stimulation protocols.
It is possible that some antivirus programs will recognize MarkovEditor as a virus. In this case, you need to explicitly instruct your antivirus program that MarkovEditor.exe is not a virus.
Furthermore, because MarkovEditor is written in Visual C++, it necessitates several dll files from the Visual C++ redistribution suite. If these are not present on the computer, an error message appears on launching MarkovEditor. In this case you need to install these dlls by executing the program vcredist86.exe, contained in the MarkovEditor master zip file. Now, MarkovEditor should run.
XXI. TUTORIAL 2: INSERTING THE 2STATE MODEL INTO MARKOVEDITOR
MarkovEditor can hold several models simultaneously, each in its own window. Windows are divided in 2 panes: the upper pane show model predictions; the lower pane harbors the model itself. All models discussed in this article are contained in the master zip file that can be downloaded from https://github.com/mikpusch/MarkovEditor.git. These models can be loaded into the program via the menu item File>Open.
The following steps describe the manual creation of the 2state model within MarkovEditor from a blank new file.

Adding the states of the model
In the lower pane, doubleclick or rightclick on empty space to create the closed and the open states. For each, assign a label (e.g., “C” to the closed state), a current value (for now, 0 for the closed state and 1 for the open state), and for now, ignore (i.e., assign default values by simply hitting ok) for sigma and initial probability function. Note that the index of state C is 0. In general, indices in MarkovEditor follow the convention of the C programming language to range from 0 to N−1 for an array of N items. A maximum of 100 states can be present in a single model.

b.
Assigning the parameters of the model
The voltagedependent 2state model is characterized by 4 parameters, \({\alpha _0}\) and \({\beta _0}\) , z, and δ (see main text). Such parameters are held in an array that is accessed by clicking on the Parameters button in the lower pane of MarkovEditor, bringing up a dialog box in which values for up to 36 parameters (from A[0] to A[35]) can be assigned.
Here, we will use in an arbitrary manner parameters A[0] for \({\alpha _0}\) , A[1] for \({\beta _0}\) , A[2] for z and A[3] for δ assigning values:

A[0] = 10

A[1] = 1, meaning that at 0 mV, the closing rate is 10 times slower than the opening rate, implying that opening is favored, i.e., that V _{1/2} is negative;

A[2] = 2, a gating valence of 2 elementary charges;

A[3] = 0.5, i.e., symmetric voltage dependence of opening and closing rate constants.
Next, to formulate the voltagedependent rate constants, we add variables, which are expressions dependent on parameters A[i], voltage (or concentration), and other variables that have been already defined. To this end, we click on the Variables button in the lower pane, select Add, and enter the following string in the dialog box

A[0]×exp(v×A[2]×A[3]/25)′ opening rate constant
creating variable w[0]. This is exactly reflecting Eq. 17, defining the voltagedependent opening rate constant (V) = \({\alpha _0}{e^{\delta zV{q_e}/\left( {kT} \right)}}\) , i.e., w[0] = α(V).
Note that expressions are not case sensitive (i.e., no distinction is made between uppercase and lowercase characters) and that comments can be added after the defining expression preceded by ′. The division by 25 (i.e., 25 mV) expresses that at ambient temperature the value of kT/q_{e} is around 25 mV. A maximum of 1,000 variables can be added to a model. We next add variable w[1] by repeating the above procedure and inserting the string

A[1]×exp(v×A[2]×(1A[3])/25)′ closing rate constant
reflecting Eq. 18, which defines the voltagedependent closing rate constant β(V) = \({\beta _0}{e^{  \left( {1  \delta } \right)zV{q_e}/\left( {kT} \right)}}\) , i.e., w[1] = β(V).
To assign these variables as rate constants for state transitions, we rightclick between the 2 states, select New transition 1 and insert the string

w[0]
in the dialog box. Next, we repeat the procedure now inserting

w[1]
as New transition 2. Then, we insert the voltagedependent singlechannel current of the open state, characterized by the singlechannel conductance γ and the reversal potential E _{rev}. We delegate these parameters to A[4] and A[5], respectively, assigning values of 10 to γ = A[4], reflecting a conductance of 10 pS, and −80 to E _{rev} = A[5] reflecting a typical equilibrium potential of −80 mV of K^{+} in excitable cells. To render this decision explicit, we define a novel variable w[2] as follows:

A[4]*(vA[5])*1e3 ' singlechannel current in pA
This expression needs 2 clarifications. First, the seemingly awkward string 1e3 (or equivalently 1E3, remember that these expressions are not case sensitive) is a convention used in C and other programming languages to enter the number 10^{−3}. For example, to enter the number 1.23*10^{4} , one would use the string 1.23E4. Second, the scaling by 10^{−3} reflects the choice to express voltage in millivolts, current in picoamperes and conductance in picosiemens. To assign this expression as the single channel current of state O, we rightclick on state O, select EditState and enter w[2] as current.
Finally, to render singlechannel simulations more realistic, i.e., to make them look like real recordings (and to test the performance of singlechannel analysis programs regarding noise), we add a noise level to each conductance state. This is used only for the stochastic simulations, not for the deterministic macroscopic predictions. To this end, we rightclick again on each state and assign a certain noise value to the sigma field, resulting in the addition of white noise with standard deviation given by the sigma value (in picoamperes). For example we add a noise of 0.05 pA to the current of the closed state and a larger value of 0.1 pA to the open state to simulate the open channel noise seen experimentally for many channels (15). Now, the 2state model is ready for exploration.
XXII. TUTORIAL 3: GENERAL SETTINGS
The general settings dialog is invoked by the menu click Settings>Edit general settings. Settings can be saved on file. Note that on startup, MarkovEditor automatically loads the latest setting file that had been saved or loaded during previous sessions.
In the dialog, the Voltage Scaling tab defines the voltage or concentration range for which steadystate properties are to be calculated and plotted. The Probability Codes tabs define the states for which the occupation probabilities is plotted. The pulse protocol consists of 3 time segments. A holding segment of duration t0, and 2 segments for which not only the voltage and concentration and duration, but also increments (or decrements) for these values can be specified. The sample time determines the number of points to be calculated (and plotted). The time step in visual simulation (ms) determines the speed with which the visual simulation of a single transporter will be visualized.
In the Spectrum tab, the frequency range over which the power spectrum is calculated is defined. No. Freq. points is number of points created for the spectrum plot. The No. Channels in stochastic simulations is relevant only for stochastic simulations! For deterministic calculations, the number of channels is considered always 1.
XXIII. TUTORIAL 4: THE PGF STIMULATION PROTOCOLS
Selecting the menu item Settings>Edit pgf file opens the pgf dialog shown in Supplemental Figure S1. The same stimulation protocols are used in the GePulse data acquisition program (https://github.com/mikpusch/GePulse.git). A sample pgf file called Pulses.pgf containing several stimulation protocols is included in the material that can be downloaded from https://github.com/mikpusch/MarkovEditor.git. Note that on startup, MarkovEditor automatically loads the latest pgf file that had been saved or loaded during previous sessions.
The most important parameter for the stimulation protocol is the sample interval (in milliseconds), which determines the density of points to be calculated. Each stimulation consists of an arbitrary number of segments, which are created by clicking on the corresponding N (for new) button and deleted by clicking on the D (for delete) button. For each segment, the segment type has to be chosen among VHOLD, CONSTANT, RAMP, SINE or WAVE. Make sure to click on the selected text after selection to make the selection definite (it has to be shown in blue color)! The meaning of the segment types are the following.
VHOLD: The initial voltage (i.e., that of the first sweep) is determined by the holding voltage, which is defined in the General Settings (see Tutorial 3). In the GePulse acquisition program, VHold is defined in the front panel. Apart from the value of the initial voltage, the segment behaves like a CONSTANT segment.
CONSTANT: The voltage remains constant throughout the segment. Its (initial) duration is determined by the Duration field. If the number of sweeps is larger than 1, the increments Delta V and Delta t are applied to determine the voltage and duration of the segment in successive sweeps. The Vfactor and tfactor are applied multiplicatively to increment (or decrement) voltage or time. In particular a tfactor of 1.5 is useful for socalled envelope protocols, such as recovery from inactivation of sodium channels. Initially, time increments are small, allowing precise sampling of the initial rapid phase and become exponentially longer, thereby reducing the number of pulses that have to be applied to reach long durations.
RAMP: The voltage changes linearly over time from the previous value (of the previous segment or VHold) to that defined in the voltage field.
SINE: The segment consists of a piece of a sine wave around the voltage defined in the voltage field, with frequency defined in the f Sine field and amplitude defined in the V sine field.
WAVE: An arbitrary shape of stimulation can be loaded from a text file. The file has to contain one number (i.e., voltage) per line. This is useful, for example, for stimulation with the shape of an action potential.
In MarkovEditor, the Leak tab, where the classical P/4 style leak subtraction can be defined, is not used. Also the Chain tab, with which in GePulse different stimulation protocols can be linked, is not used in MarkovEditor. In case the simulation is intended to be used also by the GePulse acquisition program, the Relevant Seg tab is useful to define the segment whose voltage is relevant for analysis (Rel X Seg) and the segment whose current level should be analyzed (Rel Y Seg 1).
To create a new stimulation sequence, click on an empty field on the top and assign a name. A pgf file can hold an arbitrary number of stimulation sequences, can be saved on disk by clicking on the Save button, and loaded from a disk by clicking on the Load button. Note that there is no warning if changes made have not been saved to the disk before exiting the program.
XXIV. TUTORIAL 5: GRAPH OPTIONS
For each model window, MarkovEditor keeps in memory several graphs illustrating the results of the last performed simulation. Each graph is populated once the Calculate or the VaryParams button is clicked. After changing any parameter of the model to update the graph, the Calculate or the VaryParams button has to be clicked again, i.e., graphs are not updated automatically after change of parameters.
Data shown in the graphs can be exported in ASCII format to the clipboard or to a text file by rightclicking on the graph. For some graphs (e.g., time course), data can also be exported in a binary format that can be opened by the Ana analysis program (https://github.com/mikpusch/Ana.git) and the GePulse data acquisition program (https://github.com/mikpusch/GePulse.git).
For each plot present in a graph, a short line of the same color is drawn on the top right of the graph. When the mouse pointer is placed close to such a short line, the corresponding plot will be drawn with a thick pen, and additional text related to the plot is displayed.
The axes can be in automatic or manual scale and either on a linear or a logarithmic scale. These settings can be changed by rightclicking on the axis. A variable number of ticks is shown.
XXV. TUTORIAL 6: EXPLORATION OF THE PARAMETER SPACE: THE VARY PARAMS DIALOG
MarkovEditor offers a systematic way to explore the behavior of the model when different values are assigned to the parameters encoded in the array fields A[0] … A[35]. The variation scheme is accessed by the menu item Settings>Edit vary which params. In the dialog, up to 8 parameters can be selected to be varied. For each chosen parameter, several options can be selected: a lower and an upper bound, the number of steps, and if variation should be done on a logarithmic scale. For example, to explore values 0.1, 1, and 10 for parameter A[0], the lower bound is 0.1, the upper bound 10, the number of steps 3, and variation is to be done on a logarithmic scale.
XXVI. TUTORIAL 7: ENTERING THE 3STATE, LIGANDACTIVATED CHANNEL MODEL
All models discussed in this article are contained in the master zip file that can be downloaded from https://github.com/mikpusch/MarkovEditor.git. These models can be loaded into the program via the menu item File>Open. In particular, the ligand activated 3state model is contained in the file LigandGated.mod.
However, to provide an additional example highlighting the logic behind MarkovEditor, the following steps describe the manual creation of the 3state ligand gated model within MarkovEditor from a blank new file.
The model \begin{equation}U\matrix{ {\mathop \to \limits^{\lambda \left[ L \right]} } \cr {\mathop \leftarrow \limits_\mu } \cr } B\matrix{ {\mathop \to \limits^\alpha } \cr {\mathop \leftarrow \limits_\beta } \cr } O\end{equation} is characterized by 5 parameters: λ, [L], μ, α, and β. The ligand concentration [L] could be encoded by one of the parameters, e.g., following a certain tradition of MarkovEditor, encoding it in parameter A[32]. Alternatively, and this will be done here, instead of using the voltage as the dominant variable determining channel behavior, the concentration of the ligand, denoted as c, will be used as the independent variable, playing an equivalent role as the voltage in voltagegated channels. This will allow us to easily construct doseresponse curves as well as concentration jump experiments. Parameters λ, μ, α, and β are delegated to A[0], A[1], A[2], A[3] respectively, assigning, for example, A[0]=A[1]=1 and A[2]=A[3]=2.
To create a new model, the File>New menu option can be selected and the previously mentioned parameters values are entered by clicking on the Parameters button in the lower pane. States U, B, and O are created by doubleclicking or rightclicking in empty space. As for the 2state model, to render singlechannel simulations realistic, we assign a sigma value of 0.1 to all states. The current is 0 for states U and B and 1 for state O. The ligand concentration [L] will be entered as c, instead of the variable v used for voltagegated channels. For the simple model we will not use any variable but insert the various transition rate constants by rightclicking between states and entering A[0]*c for the rate constant from state U to state B and the strings A[1], A[2], A[3] for the remaining transitions, respectively. See Supplemental Figure S2 for a screenshot of the final model.
We also adjust the general settings by choosing cLeft = 0.01, cRight = 10, deltaC = 0.01, where cLeft is the lowest, and cRight the highest concentration to be plotted in the graph displaying the steadystate concentration dependence, with points separated by steps of deltaC. We next impose the following pulse protocol Npulses=1, sample time =10, cHold = 0.01, c1=2, c2=0.01, t0=1000, t1=10000, t2=5000. This pulse emulates a concentration jump from a very low concentration (0.01 mM) to a high concentration (2 mM) and back to 0.01 mM. Do not forget to save the model (File>Save) and the settings (Settings>Save) on disk. The corresponding model file (LigandGated.mod) and settings file (LigandGated.set) can also be downloaded from https://github.com/mikpusch/MarkovEditor.git.
XXVII. TUTORIAL 8: ENTERING PATLAK MODEL OF VOLTAGEGATED NA^{+} CHANNELS—THE TEXTBASED INTERFACE
We will enter model 7 of (23) by using the textbased interface of MarkovEditor. To this end, we first create a new and empty model by selecting File>New. Now, we rightclick in the lower window, and from the popup menu, we select Edit model in text window, which will open a text window. Here, we delete all text present (rightclick and chose Select all and after that Delete from the popup menu) and paste (rightclick and select Paste from the popup menu) the following text exactly as shown:

TRANSPORTERGATING CURRENT FUNCTION:auto

FUNCTIONS:

FUNC[0]=x*a[13]/(x+a[13]) ' used to keep rate constants below upper limit a[13]

VARIABLES:

w[0]=log(6.24e12) ' ln (kT/h)  model 7 from Patlak, Physiol. Rev. 1991

w[1]=exp(w[0] + a[4] + (1a[8])*a[6] + a[7]*a[8]*v/25) ' kappa

w[2]=exp(w[0] + a[5] + a[8]*a[6]  a[7]*(1a[8])*v/25) ' lambda

w[3]=exp(w[0] + a[0] + a[7]*a[8]*v/25) ' alpha_m

w[4]=exp(w[0] + a[1]  a[7]*(1a[8])*v/25) ' bet_am

w[5]=exp(w[0] + a[2] + a[8]*a[6] + a[7]*a[8]*v/25) ' delta

w[6]=exp(w[0] + a[3] + (1a[8])*a[6]  a[7]*(1a[8])*v/25) ' gamma

w[7]=exp(w[0] + a[9]  a[11]*(1a[12])*v/25) ' alpha_h

w[8]=exp(w[0] + a[10] + a[11]*a[12]*v/25) ' beta_h

STATES:

#0;C1; i=0; sigma =0.05; initprob =1; x = 1.9089574e002; y = 0.28961749

#1;C2; i=0; sigma =0.05; initprob =1; x = 0.22393539; y = 0.29143898

#2;C3; i=0; sigma =0.05; initprob =1; x = 0.3773862; y = 0.29143898

#3;C4; i=0; sigma =0.05; initprob =1; x = 0.55506608; y = 0.29143898

#4;O; i=0.01*(v50); sigma =0.05; initprob =1; x = 0.79368576; y = 0.29326047

#5;I1; i=0; sigma =0.05; initprob =1; x = 0.55580029; y = 0.77595628

#6;I2; i=0; sigma =0.05; initprob =1; x = 0.79295154; y = 0.7704918

RATES:

FROM 0 TO 1:func[0](4*w[1])

FROM 1 TO 0:func[0](w[2])

FROM 1 TO 2:func[0](3*w[3])

FROM 2 TO 1:func[0](2*w[4])

FROM 2 TO 3:func[0](2*w[3])

FROM 3 TO 2:func[0](3*w[4])

FROM 3 TO 4:func[0](4*w[5])

FROM 3 TO 5:func[0](w[8])

FROM 4 TO 3:func[0](w[6])

FROM 4 TO 6:func[0](w[8])

FROM 5 TO 3:func[0](w[7])

FROM 5 TO 6:func[0](4*w[5]) ' 4* delta

FROM 6 TO 4:func[0](w[7])

FROM 6 TO 5:func[0](w[6])

PARAMETERS:

a[0]=19.

a[1]=22.35

a[2]=19.

a[3]=23.9

a[4]=21.32

a[5]=27.08

a[6]=1.8

a[7]=2.45

a[8]=0.6

a[9]=26.5

a[10]=22.4

a[11]=0.3

a[12]=0.5

a[13]=20000
Now, rightclick again and select Parse from the popup menu. If no error occurred (which should be the case for the previously mentioned code), a message saying Successfully parsed and copied into model should appear, signaling that everything went smoothly and that you can safely close the text window. The model should correctly appear, as illustrated in Figure 6A.
XXVIII. TUTORIAL 9: MANUAL DEFINITION OF THE GATING CURRENT FUNCTION FOR THE 2STATE MODEL
To illustrate the general use of the Transporter/gating current function, we will define a function that allows to calculate gating currents of the 2state model defined in Eq. 1 and rate constants defined in Eqs. 17 and 18.
According to Eq. 29, gating currents for the 2state model are given by \begin{equation}{I_{{\rm{gate}}}}\left( t \right) = {q_e}z\left( {{p_C}\left( t \right)\alpha  {p_O}\left( t \right)\beta } \right)\end{equation}
Thus, having implemented the 2state model in MarkovEditor, as described in Tutorial 1, using the variables w[0] and w[1] for the rate constants, and a[2] for the gating valence, the Transporter/gating current function can be inserted as

1.6e19*a[2]*(p[0]*w[0] – p[1]*w[1])/1e12 ' gating current in pA
that gives the gating current in picoamperes. This should give the same result as the automatic mode, i.e., inserting auto as the Transporter/gating current function.
XXIX. TUTORIAL 10: INSERTING THE UNIPORTER MODEL INTO MARKOVEDITOR
As in Tutorial 8, we will enter the uniporter model by using the textbased interface of MarkovEditor. To this end, we first create a new and empty model by selecting File>New. Now, we rightclick in the lower window, and from the popup menu, we select Edit model in text window, which will open a text window. Here, we delete all text present (rightclick and chose Select all and after that Delete from the popup menu) and paste (rightclick and select Paste from the popup menu) the following text exactly as shown:

TRANSPORTERGATING CURRENT FUNCTION: auto

FUNCTIONS:

VARIABLES:

w[0]=a[4]*exp(v*a[0]/2/25) ' empty transporter moves inward

w[1]=a[4]*a[7]/a[6] ' obey microscopic reversibility

w[2]=w[1]*exp(v*a[0]/2/25) ' empty transporter moves outward

w[3]=a[6]*exp(v*(a[2]+a[0])/2/25) ' loaded transporter moves inward

w[4]=a[7]*exp(v*(a[2]+a[0])/2/25) ' loaded transporter moves outward

w[5]=a[8]*exp(v*a[1]/2/25) ' unbinding of substrate to the outside

w[6]=a[8]*exp(v*(1a[1]a[2])/2/25) ' unbinding of substrate to the inside

w[7]=a[9]*a[32]*exp(v*a[1]/2/25) ' binding of substrate from outside

w[8]=a[9]*a[33]*exp(v*(1a[1]a[2])/2/25) ' binding of substrate from inside

STATES:

#0;Out0; i=0; sigma =0; initprob =1; x = 0.50848896; y = 0.81777778

#1;In 0; i=0; sigma =0; initprob =1; x = 0.52276065; y = 0.31554524

#2;Out 1; i=0; sigma =0; initprob =1; x = 0.72099853; y = 0.80046404

#3;In 1; i=0; sigma =0; initprob =1; x = 0.73568282; y = 0.32018561

RATES:

FROM 0 TO 1:w[0]

FROM 0 TO 2:w[7]

FROM 1 TO 0:w[2]

FROM 1 TO 3:w[8]

FROM 2 TO 0:w[5]

FROM 2 TO 3:w[3]

FROM 3 TO 1:w[6]

FROM 3 TO 2:w[4]

PARAMETERS:

a[0]=0.1

a[1]=0.2

a[2]=0.5

a[4]=100.

a[6]=100.

a[7]=100.

a[8]=1000.

a[9]=1000.

a[32]=1.

a[33]=1.
Now, rightclick again and select Parse from the popup menu. If no error occurred (which should be the case for the previously mentioned code), a message saying Successfully parsed and copied into model should appear, signaling that everything went smoothly and that you can safely close the text window.