diff --git a/docs/howto_update_to_v1.md b/docs/howto_update_to_v1.md index 2778d5cd5eaedc4e56e591d9aa0351fcfd3444ee..8ad320b96dc7ffd144b66ae4bcf9fd56ceae4df5 100644 --- a/docs/howto_update_to_v1.md +++ b/docs/howto_update_to_v1.md @@ -35,4 +35,61 @@ Why is this a good thing? - Now the models get loaded early! Meaning that we have a structured object at hand instead of just a string. - Now we do not need to load all backends anymore at start! The `ModelSpec` carries the `backend` name to be later loaded. - Now you can provide a json-like string as the CLI `-m` option which carries additional information about the model. A minimal string would look like `{"model_name":"openchat", "backend":"openai_compatible"}` -- Now we have a model registry that matches your given model spec with pre-defined ones. The first unifying one will be used. \ No newline at end of file +- Now we have a model registry that matches your given model spec with pre-defined ones. The first unifying one will be used. + +## No more `compute_scores` in `GameMaster`, but in `GameScorer` + +The computation of the game scores is extracted from the game master to a separate game scorer class. +This further decouples game run logic from its later evaluation. Your game code must be adjusted as follows: + +```diff +- from clemgame.clemgame import GameMaster, GameBenchmark ++ from clemgame.clemgame import GameMaster, GameBenchmark, GameScorer + +class ImageGameMaster(GameMaster): + +- def compute_scores(self, episode_interactions: Dict) -> None: +- <your scoring code> + ++ class ImageGameScorer(GameScorer): ++ ++ def __init__(self, experiment: Dict, game_instance: Dict): ++ super().__init__(GAME_NAME, experiment, game_instance) ++ self.target_grid = game_instance["target_grid"] # necessary info to score the episode + ++ def compute_scores(self, episode_interactions: Dict) -> None: ++ <your scoring code> + +class ImageGameBenchmark(GameBenchmark): + ++ def create_game_scorer(self, experiment: Dict, game_instance: Dict) -> GameScorer: ++ return ImageGameScorer(experiment, game_instance) +``` + +As you can see, the scorer will receive access to all necessary information to score an episode of the game. + +Alternatively, you can also directly migrate your code to use the `GameScorer` hooks as pre-defined in its `compute_scores` method: + +```python +class GameScorer(GameResourceLocator): + + def __init__(self, name: str, experiment: dict, game_instance: dict): + super().__init__(name) + self.experiment = experiment + self.game_instance = game_instance + """ Stores values of score computation """ + self.scores = { + "turn scores": {}, + "episode scores": {}, + } + + ... + + def compute_scores(self, episode_interactions: dict) -> None: + self.score_turns(episode_interactions) + self.score_game(episode_interactions) + + ... +``` + +Have a closer look at the `GameScorer` class for that. \ No newline at end of file