Table of contents
1. Introduction to Wumpus World
5. Agent's exploration process of the environment
2. Wumpus World Design Principles
① The single-player game part uses the pygame module:
② Computer self-playing game adopts reinforcement learning DQN algorithm
3. Wumpus world experiment code
get_random_location() function
4. Wumpus World Game Process Video Display
1. Introduction to Wumpus World
1. Wumpus World
The Wumpus world is a cave composed of multiple rooms connected together;
- Somewhere hides a Wumpus monster that will eat anyone who enters its room;
- Agent can shoot Wumpus, but only one arrow;
- Some rooms are bottomless pits, and anyone who enters these rooms will be swallowed by the bottomless pit;
The only hope of living in this environment is to find a pile of gold .
2. Problem Definition
a. Performance metrics
Climbing out of the hole with gold +1000; falling into the bottomless pit or being eaten by Wumpus -1000; taking an action -1; using arrows -10; the agent dies or the agent exits the hole, the game is over.
b.environment _
4*4 room grid. The agent starts from (1,1) and faces to the right. Gold, Wumpus, and Bottomless Pit are randomly selected in any square except the starting square ;
c. Actuator
Agent can go forward, turn left, turn right. You can pick up the objects in the grid; you can shoot arrows in the opposite direction; the arrow moves forward until it hits Wumpus; the Agent can only climb out of (1,1) .
d. Sensors
There are 5 sensors [Stink, Breeze, Golden Light, Impact, Howl ]
3. Environmental conditions
Discrete, static, single agent, partially observable
4. Main difficulties
Need to find out the bottomless pit and Wumpus, need reasoning
5. Agent's exploration process of the environment
You can refer to:
https://www.bilibili.com/video/BV147411u7So/?spm_id_from=333.337.search-card.all.click&vd_source=d77c96856e5ee7c666f69b5d4c4b6df02. Wumpus World Design Principles
1. Experimental environment
- IDE: Pycharm + Anaconda
- python：3. 7.12
- Required libraries and versions:
| pytorch |
1.13.0 |
|
| pygame |
2.1.2 |
|
| numpy |
1.18.5 |
2. Design thinking
① The single-player game part uses the pygame module:
World.py contains various operations of the room, where bottomless pits, gold, and monsters are randomly generated and not at the entrance;
Env.py (called World.py) is the overall structure of the game, including various game operation functions.
② Computer self-playing game adopts reinforcement learning DQN algorithm
Reinforcement learning DQN algorithm reward mechanism design:
| take an action | -1 | |
| Climb out of the cave with gold | +1000 | |
| Fall into a bottomless pit, eaten by Wumpus | -1000 | |
| archery |
-10 |
Note: In order to save computing resources and speed up training, the pictures are not input to the network, only the location information (hero, PIT, GOLD, WUMPUS), that is, a total of (1+PIT_COUNT +GOLD_COUNT+ WUMPUS_COUNT) neurons in the input layer
3. Operating Instructions
keyboard
Up Arrow: move up W: shoot arrow up
Down Arrow: move down S: shoot arrow down
Left Arrow: Move Left A: Shoot Arrow Left
Right arrow: move right D: shoot arrow right
mouse
Click on the cave (0,0) to exit the cave
4. Game Features
- With game difficulty selection function, four difficulty levels are set
- You can customize the number of rooms, monsters, gold piles, and bottomless pits
- It has the function of modifying the game interface and the skin of monsters and heroes
3. Wumpus world experiment code
1. world.py
Object class
Visualizations of game elements such as monsters and piles of gold. It includes functions such as loading images, resizing and setting positions.
class Object(pygame.sprite.Sprite): # 怪兽、金堆、无底洞等各种可视化图片
def __init__(self, filename, location, size=150):
# 调父类来初始化子类
pygame.sprite.Sprite.__init__(self)
# 加载图片并修改尺寸为size*size
self.image = pygame.image.load(filename).convert()
self.image = pygame.transform.smoothscale(self.image, (size, size))
self.rect = self.image.get_rect() # 获取图片位置
self.rect.topleft = location # 设置位置Room class
A room representing a labyrinth with different states such as stench, breeze, bottomless pit, gold pile and monster etc. By setting different states, the display of the room can be changed.
World class
The main logical part: it defines the size of the maze, the bottomless pit, the pile of gold, the number of monsters and arrows. By calling the set() function, you can set bottomless pits, gold piles and monsters in different positions. The shoot() function is used to shoot monsters. The set_breeze_around() and set_stench_around() functions are used to set the breeze and stench state of the surrounding room respectively.
get_random_location() function
Generate random position coordinates. It ensures that the spawned locations are not repeated and that it doesn't spawn at the maze entrance (0,0). Return a list containing the specified number of unique random positions:
def get_random_location(screen_width, screen_height, size, count=1): # 返回不重复、随机的二维数据列表, 且不包含（0 ，0），共count个元组
# random.randrange(self, start, self.stop=None, step=1, _int=int)从指定范围内按指定基数递增的集合中获取一个随机数
loc_list = []
while len(loc_list) < count:
x, y = 0, 0
while x == 0 and y == 0: # 入口（0,0）处不能有任何物体
x = random.randrange(0, screen_width, size)
y = random.randrange(0, screen_height, size)
loc = (x, y)
if loc not in loc_list:
loc_list.append(loc)
return loc_list2. env1.0.py
To be added
4. Wumpus World Game Process Video Display
Wumpus World game operation display