Jekyll2024-03-18T00:05:23-06:00https://jigang.kim/feed.xmlJigang Kimdeveloping autonomous learning agentsJigang KimDemonstration-free Autonomous Reinforcement Learning via Implicit and Bidirectional Curriculum2023-04-25T00:00:00-06:002023-04-25T00:00:00-06:00https://jigang.kim/publications/conference/ICML2023<iframe width="560" height="315" src="https://www.youtube.com/embed/OHfNI90cEPM" frameborder="0" allowfullscreen=""></iframe>
<p><strong>Abstract:</strong> While reinforcement learning (RL) has achieved great success in acquiring complex skills solely from environmental interactions, it assumes that resets to the initial state are readily available at the end of each episode. Such an assumption hinders the autonomous learning of embodied agents due to the time-consuming and cumbersome workarounds for resetting in the physical world. Hence, there has been a growing interest in autonomous RL (ARL) methods that are capable of learning from non-episodic interactions. However, existing works on ARL are limited by their reliance on prior data and are unable to learn in environments where task-relevant interactions are sparse. In contrast, we propose a demonstration-free ARL algorithm via Implicit and Bi-directional Curriculum (IBC). With an auxiliary agent that is conditionally activated upon learning progress and a bidirectional goal curriculum based on optimal transport, our method outperforms previous methods, even the ones that leverage demonstrations.</p>
<!-- -->
<h2 id="bibtex-">Bibtex <a id="bibtex"></a></h2>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>@inproceedings{kim2023demonstration,
title={Demonstration-free Autonomous Reinforcement Learning via Implicit and Bidirectional Curriculum},
author={Kim, Jigang and Cho, Daesol and Kim, H Jin},
booktitle={Proceedings of the 40th International Conference on Machine Learning},
volume={202},
number={},
pages={16441--16457},
year={2023},
publisher={PMLR}
}
</code></pre></div></div>Jigang KimWhile reinforcement learning (RL) has achieved great success in acquiring complex skills solely from environmental interactions, it assumes that resets to the initial state are readily available at the end of each episode. Such an assumption hinders the autonomous learning of embodied agents due to the time-consuming and cumbersome workarounds for resetting in the physical world. Hence, there has been a growing interest in autonomous RL (ARL) methods that are capable of learning from non-episodic interactions. However, existing works on ARL are limited by their reliance on prior data and are unable to learn in environments where task-relevant interactions are sparse. In contrast, we propose a demonstration-free ARL algorithm via Implicit and Bi-directional Curriculum (IBC). With an auxiliary agent that is conditionally activated upon learning progress and a bidirectional goal curriculum based on optimal transport, our method outperforms previous methods, even the ones that leverage demonstrations.Distributed multi-agent target search and tracking with Gaussian process and reinforcement learning2023-01-19T00:00:00-06:002023-01-19T00:00:00-06:00https://jigang.kim/publications/journal/IJCAS2023<iframe width="560" height="315" src="https://www.youtube.com/embed/mbnzpW20EWk" frameborder="0" allowfullscreen=""></iframe>
<p><strong>Abstract:</strong> Deploying multiple robots for target search and tracking has many practical applications, yet the challenge of planning over unknown or partially known targets remains difficult to address. With recent advances in deep learning, intelligent control techniques such as reinforcement learning have enabled agents to learn autonomously from environment interactions with little to no prior knowledge. Such methods can address the exploration-exploitation tradeoff of planning over unknown targets in a data-driven manner, streamlining the decision-making pipeline with end-to-end training. In this paper, we propose a multi-agent reinforcement learning technique (MARL) with target map building based on distributed Gaussian process (GP). We leverage the distributed GP to encode belief over the target locations in a scalable manner and incorporate it into centralized training with decentralized execution MARL framework to efficiently plan over unknown targets. We evaluate the performance and transferability of the trained policy in simulation and demonstrate the method on a swarm of micro unmanned aerial vehicles with hardware experiments.</p>
<!-- -->
<h2 id="bibtex-">Bibtex <a id="bibtex"></a></h2>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>@article{kim2023distributed,
title={Distributed multi-agent target search and tracking with Gaussian process and reinforcement learning},
author={Kim, Jigang and Jang, Dohyun and Kim, H Jin},
journal={International Journal of Control, Automation and Systems},
volume={21},
number={9},
pages={3057--3067},
year={2023},
publisher={Springer}
}
</code></pre></div></div>Jigang KimDeploying multiple robots for target search and tracking has many practical applications, yet the challenge of planning over unknown or partially known targets remains difficult to address. With recent advances in deep learning, intelligent control techniques such as reinforcement learning have enabled agents to learn autonomously from environment interactions with little to no prior knowledge. Such methods can address the exploration-exploitation tradeoff of planning over unknown targets in a data-driven manner, streamlining the decision-making pipeline with end-to-end training. In this paper, we propose a multi-agent reinforcement learning technique (MARL) with target map building based on distributed Gaussian process (GP). We leverage the distributed GP to encode belief over the target locations in a scalable manner and incorporate it into centralized training with decentralized execution MARL framework to efficiently plan over unknown targets. We evaluate the performance and transferability of the trained policy in simulation and demonstrate the method on a swarm of micro unmanned aerial vehicles with hardware experiments.DHRL: A Graph-Based Approach for Long-Horizon and Sparse Hierarchical Reinforcement Learning2022-09-15T00:00:00-05:002022-09-15T00:00:00-05:00https://jigang.kim/publications/conference/NEURIPS2022<iframe width="560" height="315" src="https://www.youtube.com/embed/re1zeoLHbas" frameborder="0" allowfullscreen=""></iframe>
<p><strong>Abstract:</strong> Hierarchical Reinforcement Learning (HRL) has made notable progress in complex control tasks by leveraging temporal abstraction. However, previous HRL algorithms often suffer from serious data inefficiency as environments get large. The extended components, i.e., goal space and length of episodes, impose a burden on either one or both high-level and low-level policies since both levels share the total horizon of the episode. In this paper, we present a method of Decoupling Horizons Using a Graph in Hierarchical Reinforcement Learning (DHRL) which can alleviate this problem by decoupling the horizons of high-level and low-level policies and bridging the gap between the length of both horizons using a graph. DHRL provides a freely stretchable high-level action interval, which facilitates longer temporal abstraction and faster training in complex tasks. Our method outperforms state-of-the-art HRL algorithms in typical HRL environments. Moreover, DHRL achieves long and complex locomotion and manipulation tasks.</p>
<!-- -->
<h2 id="bibtex-">Bibtex <a id="bibtex"></a></h2>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>@inproceedings{lee2022graph,
title={DHRL: A Graph-Based Approach for Long-Horizon and Sparse Hierarchical Reinforcement Learning},
author={Lee, Seungjae and Kim, Jigang and Jang, Inkyu and Kim, H Jin},
booktitle={Thirty-sixth Conference on Neural Information Processing Systems},
pages={},
year={2022},
organization={}
}
</code></pre></div></div>Jigang KimHierarchical Reinforcement Learning (HRL) has made notable progress in complex control tasks by leveraging temporal abstraction. However, previous HRL algorithms often suffer from serious data inefficiency as environments get large. The extended components, i.e., goal space and length of episodes, impose a burden on either one or both high-level and low-level policies since both levels share the total horizon of the episode. In this paper, we present a method of Decoupling Horizons Using a Graph in Hierarchical Reinforcement Learning (DHRL) which can alleviate this problem by decoupling the horizons of high-level and low-level policies and bridging the gap between the length of both horizons using a graph. DHRL provides a freely stretchable high-level action interval, which facilitates longer temporal abstraction and faster training in complex tasks. Our method outperforms state-of-the-art HRL algorithms in typical HRL environments. Moreover, DHRL achieves long and complex locomotion and manipulation tasks.Unsupervised Reinforcement Learning for Transferable Manipulation Skill Discovery2022-04-27T00:00:00-05:002022-04-27T00:00:00-05:00https://jigang.kim/publications/journal/RAL2022_2<iframe width="560" height="315" src="https://www.youtube.com/embed/bF3Y4WXfM7c" frameborder="0" allowfullscreen=""></iframe>
<p><strong>Abstract:</strong> Current reinforcement learning (RL) in robotics often experiences difficulty in generalizing to new downstream tasks due to the innate task-specific training paradigm. To alleviate it, unsupervised RL, a framework that pre-trains the agent in a task-agnostic manner without access to the task-specific reward, leverages active exploration for distilling diverse experience into essential skills or reusable knowledge. For exploiting such benefits also in robotic manipulation, we propose an unsupervised method for transferable manipulation skill discovery that ties structured exploration toward interacting behavior and transferable skill learning. It not only enables the agent to learn interaction behavior, the key aspect of the robotic manipulation learning, without access to the environment reward, but also to generalize to arbitrary downstream manipulation tasks with the learned task-agnostic skills. Through comparative experiments, we show that our approach achieves the most diverse interacting behavior and significantly improves sample efficiency in downstream tasks including the extension to multi-object, multitask problems.</p>
<!-- -->
<h2 id="bibtex-">Bibtex <a id="bibtex"></a></h2>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>@article{cho2022unsupervised,
title={Unsupervised Reinforcement Learning for Transferable Manipulation Skill Discovery},
author={Cho, Daesol and Kim, Jigang and Kim, H. Jin},
journal={IEEE Robotics and Automation Letters},
volume={7},
number={3},
pages={7455--7462},
year={2022},
publisher={IEEE}
}
</code></pre></div></div>Jigang KimCurrent reinforcement learning (RL) in robotics often experiences difficulty in generalizing to new downstream tasks due to the innate task-specific training paradigm. To alleviate it, unsupervised RL, a framework that pre-trains the agent in a task-agnostic manner without access to the task-specific reward, leverages active exploration for distilling diverse experience into essential skills or reusable knowledge. For exploiting such benefits also in robotic manipulation, we propose an unsupervised method for transferable manipulation skill discovery that ties structured exploration toward interacting behavior and transferable skill learning. It not only enables the agent to learn interaction behavior, the key aspect of the robotic manipulation learning, without access to the environment reward, but also to generalize to arbitrary downstream manipulation tasks with the learned task-agnostic skills. Through comparative experiments, we show that our approach achieves the most diverse interacting behavior and significantly improves sample efficiency in downstream tasks including the extension to multi-object, multitask problems.Automating Reinforcement Learning with Example-based Resets2022-04-04T00:00:00-05:002022-04-04T00:00:00-05:00https://jigang.kim/publications/journal/RAL2022<iframe width="560" height="315" src="https://www.youtube.com/embed/himd0Z5b64A" frameborder="0" allowfullscreen=""></iframe>
<p><strong>Abstract:</strong> Deep reinforcement learning has enabled robots to learn motor skills from environmental interactions with minimal to no prior knowledge. However, existing reinforcement learning algorithms assume an episodic setting, in which the agent resets to a fixed initial state distribution at the end of each episode, to successfully train the agents from repeated trials. Such reset mechanism, while trivial for simulated tasks, can be challenging to provide for real-world robotics tasks. Resets in robotic systems often require extensive human supervision and task-specific workarounds, which contradicts the goal of autonomous robot learning. In this paper, we propose an extension to conventional reinforcement learning towards greater autonomy by introducing an additional agent that learns to reset in a self-supervised manner. The reset agent preemptively triggers a reset to prevent manual resets and implicitly imposes a curriculum for the forward agent. We apply our method to learn from scratch on a suite of simulated and real-world continuous control tasks and demonstrate that the reset agent successfully learns to reduce manual resets whilst also allowing the forward policy to improve gradually over time.</p>
<!-- -->
<h2 id="bibtex-">Bibtex <a id="bibtex"></a></h2>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>@article{kim2022automating,
title={Automating Reinforcement Learning With Example-Based Resets},
author={Kim, Jigang and Park, J. hyeon and Cho, Daesol and Kim, H. Jin},
journal={IEEE Robotics and Automation Letters},
volume={7},
number={3},
pages={6606--6613},
year={2022},
publisher={IEEE}
}
</code></pre></div></div>Jigang KimDeep reinforcement learning has enabled robots to learn motor skills from environmental interactions with minimal to no prior knowledge. However, existing reinforcement learning algorithms assume an episodic setting, in which the agent resets to a fixed initial state distribution at the end of each episode, to successfully train the agents from repeated trials. Such reset mechanism, while trivial for simulated tasks, can be challenging to provide for real-world robotics tasks. Resets in robotic systems often require extensive human supervision and task-specific workarounds, which contradicts the goal of autonomous robot learning. In this paper, we propose an extension to conventional reinforcement learning towards greater autonomy by introducing an additional agent that learns to reset in a self-supervised manner. The reset agent preemptively triggers a reset to prevent manual resets and implicitly imposes a curriculum for the forward agent. We apply our method to learn from scratch on a suite of simulated and real-world continuous control tasks and demonstrate that the reset agent successfully learns to reduce manual resets whilst also allowing the forward policy to improve gradually over time.Integrated simulator and hardware controller package for UR3 manipulator2022-01-01T00:00:00-06:002022-01-01T00:00:00-06:00https://jigang.kim/blog/ur3-mujoco-env<p>This <a href="https://github.com/snu-larr/dual-ur3-env/releases">integrated package</a> provides <a href="https://mujoco.org/">MuJoCo</a>-based simulator and hardware controller based on <a href="https://www.universal-robots.com/download/manuals-e-series/script/script-manual-e-series-sw-511/">URScript API</a> for UR3 Universal Robots.</p>
<p>The package was primarily developed for the LARR dual-UR3 robot hardware.
LARR dual-UR3 robot consists of two <a href="https://www.universal-robots.com/products/ur3-robot/">Universal Robots UR3</a> manipulators each equipped with <a href="https://robotiq.com/products/2f85-140-adaptive-robot-gripper">Robotiq 2F-85</a> gripper mounted downwards at a 45-degree angle to a customized aluminum-frame stand built by <a href="https://vention.io/">Vention</a>.</p>
<p><img src="/images/posts/dual-ur3-mujoco/all.png" alt="png" /></p>Jigang KimThis integrated package provides MuJoCo-based simulator and hardware controller based on URScript API for UR3 Universal Robots.Spatio-semantic Task Recognition: Unsupervised Learning of Task-discriminative Features for Segmentation and Imitation2021-09-02T00:00:00-05:002021-09-02T00:00:00-05:00https://jigang.kim/publications/journal/IJCAS2021<!-- <iframe width="560" height="315" src="https://www.youtube.com/embed/U-wDbIs1ePY" frameborder="0" allowfullscreen></iframe>
-->
<p><strong>Abstract:</strong> Discovering task subsequences from a continuous video stream facilitates a robot imitation of sequential tasks. In this research, we develop unsupervised learning of the task subsequences which does not require a human teacher to give the supervised label of the subsequence. Task-discriminative feature, in the form of sparsely activated cells called task capsules, is proposed for self-training to preserve spatio-semantic information of a visual input. The task capsules are sparsely and exclusively activated with respect to the spatio-semantic context of the task subsequence: a type and location of the object. Therefore, the generalized purpose in multiple videos is unsupervisedly discovered according to the spatio-semantic context, and the demonstration is segmented into the task subsequences in an object-centric way. In comparison with the existing studies on unsupervised task segmentation, our work has the following distinct contribution: 1) the task provided as a video stream can be segmented without any pre-defined knowledge, 2) the trained features preserve spatio-semantic information so that the segmentation is object-centric. Our experiment shows that the recognition of the task subsequence can be applied to robot imitation for a sequential pick-and-place task by providing the semantic and location information of the object to be manipulated.</p>
<!-- -->
<h2 id="bibtex-">Bibtex <a id="bibtex"></a></h2>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>@article{park2021spatio,
title={Spatio-semantic Task Recognition: Unsupervised Learning of Task-discriminative Features for Segmentation and Imitation},
author={Park, J Hyeon and Kim, Jigang and Kim, H Jin},
journal={International Journal of Control, Automation and Systems},
volume={19},
number={10},
pages={3409--3418},
year={2021},
publisher={Springer}
}
</code></pre></div></div>Jigang KimDiscovering task subsequences from a continuous video stream facilitates a robot imitation of sequential tasks. In this research, we develop unsupervised learning of the task subsequences which does not require a human teacher to give the supervised label of the subsequence. Task-discriminative feature, in the form of sparsely activated cells called task capsules, is proposed for self-training to preserve spatio-semantic information of a visual input. The task capsules are sparsely and exclusively activated with respect to the spatio-semantic context of the task subsequence: a type and location of the object. Therefore, the generalized purpose in multiple videos is unsupervisedly discovered according to the spatio-semantic context, and the demonstration is segmented into the task subsequences in an object-centric way. In comparison with the existing studies on unsupervised task segmentation, our work has the following distinct contribution: 1) the task provided as a video stream can be segmented without any pre-defined knowledge, 2) the trained features preserve spatio-semantic information so that the segmentation is object-centric. Our experiment shows that the recognition of the task subsequence can be applied to robot imitation for a sequential pick-and-place task by providing the semantic and location information of the object to be manipulated.Running OpenVPN on an isolated network namespace2021-05-17T00:00:00-05:002021-05-17T00:00:00-05:00https://jigang.kim/blog/vpn-on-network-namespace<p>It may be convenient to run select processes over VPN instead of applying the VPN connection system-wide.
<a href="https://github.com/crasm/vpnshift.sh">vpnshift</a> by GitHub user <a href="https://github.com/crasm">crasm</a> lets you run commands in an isolated network namespace with OpenVPN.
Updates were made to crasm’s original repository in <a href="https://github.com/jigangkim/vpnshift.sh">this forked version</a> with additional funtionalities and command line options.</p>
<p><strong>Outline</strong></p>
<ul>
<li><a href="#usage">Usage</a></li>
<li><a href="#scripts">Example scripts</a></li>
</ul>
<h2 id="usage">Usage</h2>
<p>Download <a href="https://github.com/jigangkim/vpnshift.sh/blob/master/vpnshift">this vpnshift bash script</a> and follow the instructions below:</p>
<ol>
<li>
<p>Apply VPN to a single process(COMMAND).</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="nb">sudo </span>bash ./vpnshift <span class="nt">-c</span> <OVPN FILE> <span class="nt">-i</span> <INTERFACE> <span class="nt">-u</span> <USER> <span class="o">[</span><COMMAND> <span class="o">[</span><ARG>...]]
</code></pre></div> </div>
</li>
<li>
<p>Establish a VPN network namespace with a dummy ping command.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="nb">sudo </span>bash ./vpnshift <span class="nt">-c</span> <OVPN FILE> <span class="nt">-i</span> <INTERFACE> <span class="nt">-u</span> <USER> ping google.com
</code></pre></div> </div>
<p>..and attach multiple processes to it.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="nb">sudo </span>ip netns <span class="nb">exec</span> <NAMESPACE> <span class="nb">sudo</span> <span class="nt">-u</span> <USER> <span class="o">[</span><COMMAND> <span class="o">[</span><ARG>...]]
</code></pre></div> </div>
</li>
</ol>
<h2 id="example-scripts">Example scripts</h2>
<p>Bash script to run firefox on an existing network namespace:</p>
<p><strong>firefox_vpnshift.sh</strong></p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">usage</span><span class="o">=</span><span class="s2">"usage: bash firefox_vpnshift.sh -n <netns>"</span>
quick_die<span class="o">()</span> <span class="o">{</span>
<span class="nv">format</span><span class="o">=</span><span class="s2">"</span><span class="nv">$1</span><span class="s2">"</span><span class="p">;</span> <span class="nb">shift</span>
<span class="o">></span>&2 <span class="nb">printf</span> <span class="s2">"</span><span class="k">${</span><span class="nv">format</span><span class="k">}</span><span class="se">\n</span><span class="s2">"</span> <span class="s2">"</span><span class="nv">$@</span><span class="s2">"</span>
<span class="nb">exit </span>1
<span class="o">}</span>
main<span class="o">()</span> <span class="o">{</span>
<span class="k">while </span><span class="nb">getopts</span> <span class="s2">"hn:"</span> opt<span class="p">;</span> <span class="k">do
case</span> <span class="s2">"</span><span class="k">${</span><span class="nv">opt</span><span class="k">}</span><span class="s2">"</span> <span class="k">in
</span>h<span class="p">)</span> quick_die <span class="s2">"</span><span class="k">${</span><span class="nv">usage</span><span class="k">}</span><span class="s2">"</span> <span class="p">;;</span>
n<span class="p">)</span> <span class="nv">netns</span><span class="o">=</span><span class="s2">"</span><span class="k">${</span><span class="nv">OPTARG</span><span class="k">}</span><span class="s2">"</span> <span class="p">;;</span>
<span class="k">*</span><span class="p">)</span> quick_die <span class="s2">"unknown option: %s"</span> <span class="s2">"</span><span class="k">${</span><span class="nv">opt</span><span class="k">}</span><span class="s2">"</span> <span class="p">;;</span>
<span class="k">esac</span>
<span class="k">done
</span><span class="nb">shift</span> <span class="k">$((</span> OPTIND <span class="o">-</span> <span class="m">1</span> <span class="k">))</span>
<span class="k">if</span> <span class="o">[[</span> <span class="nt">-z</span> <span class="s2">"</span><span class="k">${</span><span class="nv">netns</span><span class="k">}</span><span class="s2">"</span> <span class="o">]]</span><span class="p">;</span> <span class="k">then
</span>quick_die <span class="s2">"network namespace is required"</span>
<span class="k">fi
</span><span class="nb">sudo </span>ip netns <span class="nb">exec</span> <span class="nv">$netns</span> <span class="nb">sudo</span> <span class="nt">-u</span> <span class="nv">$USER</span> firefox
<span class="o">}</span>
main <span class="s2">"</span><span class="nv">$@</span><span class="s2">"</span>
</code></pre></div></div>Jigang KimIt may be convenient to run select processes over VPN instead of applying the VPN connection system-wide. vpnshift by GitHub user crasm lets you run commands in an isolated network namespace with OpenVPN. Updates were made to crasm’s original repository in this forked version with additional funtionalities and command line options.Replace GitHub personal access token with a simple password2021-05-06T00:00:00-05:002021-05-06T00:00:00-05:00https://jigang.kim/blog/replace-github-token-with-password<p>GitHub has announced <a href="https://github.blog/2020-12-15-token-authentication-requirements-for-git-operations/">plans</a> to disable account passwords when authenticating Git operations such as <strong>git clone</strong> and <strong>git push</strong>.
Beginning August 13, 2021, GitHub will require token-based authentication.
The following script aims to restore the password authentication functionality (though it requires a one-time setup) for users who want to operate business as usual:</p>
<p><strong>Outline</strong></p>
<ul>
<li><a href="#encode-token">Encoding your GitHub token</a></li>
<li><a href="#setup">Setup script</a></li>
<li><a href="#bash-alias">Create a bash alias</a></li>
</ul>
<h2 id="encoding-your-github-token">Encoding your GitHub token</h2>
<p>Encode your personal access token by running the following script.
The script depends on the OpenSSL package which you may need to install through <a href="https://wiki.debian.org/Apt"><strong>apt</strong></a> or other package managers.</p>
<p><strong>encode.sh</strong></p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">#!/usr/bin/env bash</span>
<span class="nb">echo</span> <span class="nt">-n</span> <span class="s2">"Token: "</span>
<span class="nb">read</span> <span class="nt">-s</span> token
<span class="nb">echo
echo</span> <span class="nt">-n</span> <span class="s2">"Enter password: "</span>
<span class="nb">read</span> <span class="nt">-s</span> password1
<span class="nb">echo
echo</span> <span class="nt">-n</span> <span class="s2">"Retype password: "</span>
<span class="nb">read</span> <span class="nt">-s</span> password2
<span class="nb">echo
</span><span class="k">if</span> <span class="o">[[</span> <span class="s2">"</span><span class="nv">$password1</span><span class="s2">"</span> <span class="o">==</span> <span class="s2">"</span><span class="nv">$password2</span><span class="s2">"</span> <span class="o">]]</span><span class="p">;</span> <span class="k">then
</span><span class="nb">echo</span> <span class="nt">-n</span> <span class="s2">"Encrypted token: "</span>
<span class="nb">echo</span> <span class="nv">$token</span> | openssl aes-256-cbc <span class="nt">-pbkdf2</span> <span class="nt">-a</span> <span class="nt">-salt</span> <span class="nt">-pass</span> pass:<span class="nv">$password1</span>
<span class="k">else
</span><span class="nb">echo</span> <span class="s2">"Password does not match!"</span>
<span class="k">fi</span>
</code></pre></div></div>
<p>Make sure that your token has been properly encoded by checking whether decoding the encrypted token recovers the original token with the following script:</p>
<p><strong>decode.sh</strong></p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">#!/usr/bin/env bash</span>
<span class="nb">echo</span> <span class="nt">-n</span> <span class="s2">"Encrypted token: "</span>
<span class="nb">read </span>secret
<span class="nb">echo</span> <span class="nt">-n</span> <span class="s2">"Password: "</span>
<span class="nb">read</span> <span class="nt">-s</span> password
<span class="nb">echo
echo</span> <span class="nt">-n</span> <span class="s2">"Token: "</span>
<span class="nb">echo</span> <span class="nv">$secret</span> | openssl aes-256-cbc <span class="nt">-pbkdf2</span> <span class="nt">-d</span> <span class="nt">-a</span> <span class="nt">-pass</span> pass:<span class="nv">$password</span>
</code></pre></div></div>
<h2 id="setup-script">Setup script</h2>
<p>Now you can replace your encryted token in the following script and run the script.
The script automatically sets up environment variables GIT_ASKPASS and GIT_TOKEN for the current terminal session.
You will be prompted to enter the password you used to encode your token.
Then you can run GitHub operations from the terminal as usual!</p>
<p><strong>github-setup.sh</strong></p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">#!/usr/bin/env bash</span>
<span class="nv">secret</span><span class="o">=</span><your encryted github token goes here>
<span class="nb">echo</span> <span class="nt">-n</span> <span class="s2">"Password: "</span>
<span class="nb">read</span> <span class="nt">-s</span> password
<span class="nb">echo
</span><span class="nv">token</span><span class="o">=</span><span class="sb">`</span><span class="nb">echo</span> <span class="nv">$secret</span> | openssl aes-256-cbc <span class="nt">-pbkdf2</span> <span class="nt">-d</span> <span class="nt">-a</span> <span class="nt">-pass</span> pass:<span class="nv">$password</span><span class="sb">`</span>
<span class="nb">echo</span> <span class="s1">'echo $GIT_TOKEN'</span> <span class="o">></span> <span class="nv">$HOME</span>/.git-askpass
<span class="nb">chmod</span> +x <span class="nv">$HOME</span>/.git-askpass
<span class="nb">export </span><span class="nv">GIT_TOKEN</span><span class="o">=</span><span class="nv">$token</span>
<span class="nb">export </span><span class="nv">GIT_ASKPASS</span><span class="o">=</span><span class="nv">$HOME</span>/.git-askpass
</code></pre></div></div>
<h2 id="create-a-bash-alias">Create a bash alias</h2>
<p>For easy access, assign a shortcut command for the <strong>github-setup.sh</strong> script with bash alias.
Look for the <strong>.bash_aliases</strong> file in your home directory. If it doesn’t exist, make a empty file.
Append the following line to the file:</p>
<p><strong>.bash_aliases</strong></p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">alias </span><span class="nv">gitoken</span><span class="o">=</span><span class="s1">'source /<base directory to the script>/github-setup.sh'</span>
</code></pre></div></div>
<p>Now you can enter the command <strong>gitoken</strong> from anywhere in the terminal to run the script.
You can replace the alias <strong>gitoken</strong> with an alias of your liking.</p>Jigang KimGitHub has announced plans to disable account passwords when authenticating Git operations such as git clone and git push. Beginning August 13, 2021, GitHub will require token-based authentication. The following script aims to restore the password authentication functionality (though it requires a one-time setup) for users who want to operate business as usual:Zero-Shot Transfer Learning of a Throwing Task via Domain Randomization2020-10-13T00:00:00-05:002020-10-13T00:00:00-05:00https://jigang.kim/publications/conference/ICCAS2020<p><strong>Abstract:</strong> Deep reinforcement learning (DRL) on continuous robot control has received a wide range of interests over the last decade. Collecting data directly from real robots results in high sample complexities and can cause safety accidents, so simulators are widely used as efficient alternatives for real robots. Unfortunately, policies trained in the simulation cannot be directly transferred to real-world robots due to a mismatch between the simulation and the reality, which is referred to as ‘reality gap’. To close this gap, domain randomization (DR) is commonly used. DR guarantees better transferability in the zero-shot setting, i.e. training agents in the source domain and testing them on the previously unseen target domain without fine-tuning. In this work, we identify the positive influence of DR on zero-shot transfer between the sim-to-sim setting with an object throwing task.</p>
<!-- <iframe width="560" height="315" src="https://www.youtube.com/embed/U-wDbIs1ePY" frameborder="0" allowfullscreen></iframe>
-->
<!-- -->
<h2 id="bibtex-">Bibtex <a id="bibtex"></a></h2>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>@inproceedings{park2020zero,
title={Zero-shot transfer learning of a throwing task via domain randomization},
author={Park, Sungyong and Kim, Jigang and Kim, H Jin},
booktitle={2020 20th International Conference on Control, Automation and Systems (ICCAS)},
pages={1026--1030},
year={2020},
organization={IEEE}
}
</code></pre></div></div>Jigang KimDeep reinforcement learning (DRL) on continuous robot control has received a wide range of interests over the last decade. Collecting data directly from real robots results in high sample complexities and can cause safety accidents, so simulators are widely used as efficient alternatives for real robots. Unfortunately, policies trained in the simulation cannot be directly transferred to real-world robots due to a mismatch between the simulation and the reality, which is referred to as ‘reality gap’. To close this gap, domain randomization (DR) is commonly used. DR guarantees better transferability in the zero-shot setting, i.e. training agents in the source domain and testing them on the previously unseen target domain without fine-tuning. In this work, we identify the positive influence of DR on zero-shot transfer between the sim-to-sim setting with an object throwing task.