Skip to content
Snippets Groups Projects
Commit 8fc8ea05 authored by Philipp Sadler's avatar Philipp Sadler
Browse files

issue #40: add docu to adjust game code

parent 493f6ae1
No related branches found
No related tags found
No related merge requests found
...@@ -35,4 +35,61 @@ Why is this a good thing? ...@@ -35,4 +35,61 @@ Why is this a good thing?
- Now the models get loaded early! Meaning that we have a structured object at hand instead of just a string. - Now the models get loaded early! Meaning that we have a structured object at hand instead of just a string.
- Now we do not need to load all backends anymore at start! The `ModelSpec` carries the `backend` name to be later loaded. - Now we do not need to load all backends anymore at start! The `ModelSpec` carries the `backend` name to be later loaded.
- Now you can provide a json-like string as the CLI `-m` option which carries additional information about the model. A minimal string would look like `{"model_name":"openchat", "backend":"openai_compatible"}` - Now you can provide a json-like string as the CLI `-m` option which carries additional information about the model. A minimal string would look like `{"model_name":"openchat", "backend":"openai_compatible"}`
- Now we have a model registry that matches your given model spec with pre-defined ones. The first unifying one will be used. - Now we have a model registry that matches your given model spec with pre-defined ones. The first unifying one will be used.
\ No newline at end of file
## No more `compute_scores` in `GameMaster`, but in `GameScorer`
The computation of the game scores is extracted from the game master to a separate game scorer class.
This further decouples game run logic from its later evaluation. Your game code must be adjusted as follows:
```diff
- from clemgame.clemgame import GameMaster, GameBenchmark
+ from clemgame.clemgame import GameMaster, GameBenchmark, GameScorer
class ImageGameMaster(GameMaster):
- def compute_scores(self, episode_interactions: Dict) -> None:
- <your scoring code>
+ class ImageGameScorer(GameScorer):
+
+ def __init__(self, experiment: Dict, game_instance: Dict):
+ super().__init__(GAME_NAME, experiment, game_instance)
+ self.target_grid = game_instance["target_grid"] # necessary info to score the episode
+ def compute_scores(self, episode_interactions: Dict) -> None:
+ <your scoring code>
class ImageGameBenchmark(GameBenchmark):
+ def create_game_scorer(self, experiment: Dict, game_instance: Dict) -> GameScorer:
+ return ImageGameScorer(experiment, game_instance)
```
As you can see, the scorer will receive access to all necessary information to score an episode of the game.
Alternatively, you can also directly migrate your code to use the `GameScorer` hooks as pre-defined in its `compute_scores` method:
```python
class GameScorer(GameResourceLocator):
def __init__(self, name: str, experiment: dict, game_instance: dict):
super().__init__(name)
self.experiment = experiment
self.game_instance = game_instance
""" Stores values of score computation """
self.scores = {
"turn scores": {},
"episode scores": {},
}
...
def compute_scores(self, episode_interactions: dict) -> None:
self.score_turns(episode_interactions)
self.score_game(episode_interactions)
...
```
Have a closer look at the `GameScorer` class for that.
\ No newline at end of file
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment