openAI gym API 관련 예제

openAI에서는 여러가지 개발환경에 대한 리소스를 제공하고 있다.

특히 AI 분석환경 시뮬레이션 소스코드도 제공하기 때문에 이에 대한 활용도 가능하다.

 

https://robotics.farama.org/

https://robotics.farama.org/envs/fetch/pick_and_place/

Gymnasium-Robotics Documentation

위 링크는 로봇에 대한 시뮬레이션 환경 제공 링크이다. 

 

import gymnasium as gym
env = gym.make("FetchPickAndPlace-v2", render_mode="human")
observation, info = env.reset(seed=42)
for _ in range(1000):
   action = policy(observation)  # User-defined policy function
   observation, reward, terminated, truncated, info = env.step(action)

   if terminated or truncated:
      observation, info = env.reset()
env.close()

예제를 제공하지만 policy(정책) 에 대한 정의는 사전에 

코드를 작성하는 것이 필요하다. 

 

관련 논문은 아래 링크를 참고하면 된다.

https://arxiv.org/pdf/1802.09464.pdf

 

총 24가지의 상태값이 있어서 이 부분들을 학습 시 변수로 입력해야 한다.

 

NumObservationMinMaxSite Name (in corresponding XML file)Joint Name (in corresponding XML file)Joint TypeUnit

0	End effector x position in global coordinates	-Inf	Inf	robot0:grip	-	-	position (m)
1	End effector y position in global coordinates	-Inf	Inf	robot0:grip	-	-	position (m)
2	End effector z position in global coordinates	-Inf	Inf	robot0:grip	-	-	position (m)
3	Block x position in global coordinates	-Inf	Inf	object0	-	-	position (m)
4	Block y position in global coordinates	-Inf	Inf	object0	-	-	position (m)
5	Block z position in global coordinates	-Inf	Inf	object0	-	-	position (m)
6	Relative block x position with respect to gripper x position in globla coordinates. Equals to xgripper - xblock	-Inf	Inf	object0	-	-	position (m)
7	Relative block y position with respect to gripper y position in globla coordinates. Equals to ygripper - yblock	-Inf	Inf	object0	-	-	position (m)
8	Relative block z position with respect to gripper z position in globla coordinates. Equals to zgripper - zblock	-Inf	Inf	object0	-	-	position (m)
9	Joint displacement of the right gripper finger	-Inf	Inf	-	robot0:r_gripper_finger_joint	hinge	position (m)
10	Joint displacement of the left gripper finger	-Inf	Inf	-	robot0:l_gripper_finger_joint	hinge	position (m)
11	Global x rotation of the block in a XYZ Euler frame rotation	-Inf	Inf	object0	-	-	angle (rad)
12	Global y rotation of the block in a XYZ Euler frame rotation	-Inf	Inf	object0	-	-	angle (rad)
13	Global z rotation of the block in a XYZ Euler frame rotation	-Inf	Inf	object0	-	-	angle (rad)
14	Relative block linear velocity in x direction with respect to the gripper	-Inf	Inf	object0	-	-	velocity (m/s)
15	Relative block linear velocity in y direction with respect to the gripper	-Inf	Inf	object0	-	-	velocity (m/s)
16	Relative block linear velocity in z direction	-Inf	Inf	object0	-	-	velocity (m/s)
17	Block angular velocity along the x axis	-Inf	Inf	object0	-	-	angular velocity (rad/s)
18	Block angular velocity along the y axis	-Inf	Inf	object0	-	-	angular velocity (rad/s)
19	Block angular velocity along the z axis	-Inf	Inf	object0	-	-	angular velocity (rad/s)
20	End effector linear velocity x direction	-Inf	Inf	robot0:grip	-	-	velocity (m/s)
21	End effector linear velocity y direction	-Inf	Inf	robot0:grip	-	-	velocity (m/s)
22	End effector linear velocity z direction	-Inf	Inf	robot0:grip	-	-	velocity (m/s)
23	Right gripper finger linear velocity	-Inf	Inf	-	robot0:r_gripper_finger_joint	hinge	velocity (m/s)
24	Left gripper finger linear velocity	-Inf	Inf	-	robot0:l_gripper_finger_joint	hinge	velocity (m/s)

 

 

 

참고 : 강화학습에 필요한 모듈인 baselines 를 설치시 stable-Baselines3 Docs를 꼭 읽어보길 바란다...

https://stable-baselines3.readthedocs.io/en/master/

Stable-Baselines3 Docs - Reliable Reinforcement Learning Implementations — Stable Baselines3 2.0.0a11 documentation