# System evaluation

This page is meant to provide a short introduction to the evaluation of a system with characteristic "systemic", i.e. non-linear behavior. The system is modeled with Netlogo and analyzed with Netlogo's BehaviorSpace-tool and evaluated and visualized with Python.

## The bushfire model

The bushfire model is an often used example for demonstrating critical phase transitions in complex systems. The model considers the propagation of a bushfire which critically depends on the density of trees. While loosely dispersed bushes hamper the propagation, densely growing trees can cause exhaustive devastation. The transition from just regional, small fires to large and global fires however does not proceed linearly with increasing density. It proceeds non-linearly with the typical sigmoid form of a rapid phase phase transition.

In the model, the fire is assumed to spread from the left. It propagates to adjacent bushes (green patches) with a probability of 0.33. The pictures below show the spread of fire (red patches) in three simulated forests, one with a density of 25% (left) - about 10 % of trees catch fire in this case -, one with a density of 37% (middle) - about 19 % of trees catch fire -, and one with a density of 50% (right) with 91 % of all trees burning.

Click on the image to access a Netlogo/Tortoise implementation of the model.

The code for this simplified Netlogo model can be downloaded here. (For a slightly more elaborate model see the fire-model in Netlogo's Models Library)

If we run this model with a parameter sweep, that is, with densities gradually increasing from 1 to 100%. and plot the results (below), we can sense that the percentage of trees burned might rise rather slowly initially. Than, from about 30% density onwards the fraction of burned trees ascends steeply to reach 90% already at about 45% density. The remaining  10% of  burned trees than seem to be gained rather slowly again.

However, due to random variations in the positions of bushes in the test-runs and also due to the fact that adjacent bushes catch fire only with a probability of 0.33, the result has a high variance, indicating a rapid phase transition but not proofing it unmistakably.

To see the effect more clearly, one would have to run this test more often and compute a statistical mean of the results.

### Behavior space

Netlogo offers an analytical tool, called BehaviorSpace ( > Menu > Tools > BehaviorSpace), for testing a model statistically in several runs. Opening this tool and pushing the button Edit in the ensuing window (below) provides a self-explaining input dialog (right), which in the case of this example could be filled as shown, generating conditions for a test run with 100 repetitions.

What BehaviorSpace generates, is either a spreadsheet-file or a table-file both of csv-format. While the former tries to be "human readable" to some extent, the later is more appropriate for generating large data sets. Both of these files can be opened with a common spreadsheet-program (such as MS-Excel for instance). To gain more flexibility however, it seems recommendable to access data in more direct form. The following (simplified and heavily annotated) Python-code is meant to support first learning steps for the analysis and the plotting of the results from Netlogo's BehaviorSpace runs.

The first part of the code below imports data from the csv-file and brings it into the format that Python's data analytic library Pandas can process.

The next part below provides the number of repetitions of the BehaviorSpace run. Since the Netlogo-generated file does not indicate repetitions explicitly, we have to make Python count the number of times that certain densities occur in the file.

This last part finally calculate the mean of the repetitions and plots it.

The output of a parameter sweep with each density averaged over 100 repetitions, now shows more clearly that the main part of the ascend towards 100% burned trees is compressed into a small range between 30% and 47% of density (red lines).