Releases: microsoft/UFO
v0.1.1
Highlights
New Release for v0.1.1! We've made some significant updates! Previously known as AppAgent and ActAgent, we've rebranded them to HostAgent and AppAgent to better align with their functionalities. Explore the latest enhancements:
1. Learning from Human Demonstration: UFO now supports learning from human demonstration! Utilize the Windows Step Recorder to record your steps and demonstrate them for UFO. Refer to our detailed guide in README.md for more information.
2. Win32 Support: We've incorporated support for Win32 as a control backend, enhancing our UI automation capabilities.
3. Extended Application Interaction: UFO now goes beyond UI controls, allowing interaction with your application through keyboard inputs and native APIs! Presently, we support Word (examples), with more to come soon. Customize and build your own interactions.
4. Control Filtering: Streamline LLM's action process by using control filters to remove irrelevant control items. Enable them in config_dev.yaml under the control filtering section at the bottom.
What's Changed
- et_text enabled by @Lenny2Liu in #35
- price by @Mac0q in #42
- Enable human demonstration RAG by @yunhao0204 in #49
- Bump pillow from 10.2.0 to 10.3.0 by @dependabot in #52
- Vyokky/dev Agent and automator modularization + Learning for demonstration by @vyokky in #50
- update function name and add demo video in README by @yunhao0204 in #54
- hosted api by @Mac0q in #55
- enable win32 backend support by @yunhao0204 in #59
- Update README.md by @vyokky in #60
- control filter by @Mac0q in #61
- New openfileapp by @Lenny2Liu in #57
- fix filter by @Mac0q in #62
- Code refactor and control filtered by @vyokky in #63
- Minor fix for releasing new version by @vyokky in #67
- Dev/reformat by @vyokky in #69
- New release for v0.2.0 by @vyokky in #70
New Contributors
- @dependabot made their first contribution in #52
Full Changelog: v0.0.1...v0.2.0
What's Changed
- et_text enabled by @Lenny2Liu in #35
- price by @Mac0q in #42
- Enable human demonstration RAG by @yunhao0204 in #49
- Bump pillow from 10.2.0 to 10.3.0 by @dependabot in #52
- Vyokky/dev Agent and automator modularization + Learning for demonstration by @vyokky in #50
- update function name and add demo video in README by @yunhao0204 in #54
- hosted api by @Mac0q in #55
- enable win32 backend support by @yunhao0204 in #59
- Update README.md by @vyokky in #60
- control filter by @Mac0q in #61
- New openfileapp by @Lenny2Liu in #57
- fix filter by @Mac0q in #62
- Code refactor and control filtered by @vyokky in #63
- Minor fix for releasing new version by @vyokky in #67
- Dev/reformat by @vyokky in #69
- New release for v0.2.0 by @vyokky in #70
New Contributors
- @dependabot made their first contribution in #52
Full Changelog: v0.0.1...v0.1.1
Contributors
Assets 2
v0.0.1
What's Changed
Key Updates:
1. We now support creating your help documents for each Windows application to become an app expert. Check the [README](./learner/README.md) for more details!
2. UFO now supports RAG from offline documents and online Bing search.
3. You can save the task completion trajectory into its memory for UFO's reference, improving its future success rate!
4. You can customize different GPT models for AppAgent and ActAgent. Text-only models (e.g., GPT-4) are now supported!
Detailed contributions
- Update README.md by @eltociear in #1
- fix: word spelling in prompt by @saifeiLee in #6
- Fix colorama init by @lserinol in #10
- Modify the configuration of AOAI API base and modify the dict of app_selection prompt by @kangyu in #12
- Add config file to ignore, and added a template config file for AAD login method by @Mac0q in #16
- Develop the RAG with offline doc and online bing search by @vyokky in #14
- Develop reformating prompts and readme for offline learner by @vyokky in #19
- readme by @vyokky in #23
- Vyokky/dev: Support Non-visual mode by @vyokky in #24
- OpenAI API by @Mac0q in #25
- check in openfile.py by @Lenny2Liu in #26
- Vyokky/dev Experience Learning by @vyokky in #28
- Add record processor to parse the user behavior record to JSON file by @yunhao0204 in #31
- openai config by @Mac0q in #32
- Fix the prompts by @AL-377 in #33
- Vyokky/executor by @vyokky in #34
- Vyokky/dev v0.0.1 New release by @vyokky in #36
- release v0.0.1 by @vyokky in #38
New Contributors
- @eltociear made their first contribution in #1
- @saifeiLee made their first contribution in #6
- @lserinol made their first contribution in #10
- @kangyu made their first contribution in #12
- @Mac0q made their first contribution in #16
- @vyokky made their first contribution in #14
- @Lenny2Liu made their first contribution in #26
- @AL-377 made their first contribution in #33
Full Changelog: https://github.com/microsoft/UFO/commits/v0.0.1
