Progress Report: June 2017
https://rpcs3.net/blog/2017/07/03/progress-report-june-2017
Another month another colossal RPCS3 progress record. But first things first, take a look at this beautiful introduction to RPCS3 video that reznoire made:
Moving on to some statistics the numbers alone this month are quite impressive, and that is not even counting games that improved from emulation improvements that are still work in progress.
Looking at the GitHub statistics 16 authors have pushed 140 commits to the master branch. Here 353 files have changed and there have been 19,729 additions and 17,660 deletions. These numbers are much larger than normal, and several exciting changes are behind it. This month RPCS3 transitioned away from the previous GUI toolkit Wx to Qt which in turn also lead to several user facing interface improvements. This separate report goes into significantly greater detail regarding what work has already been done on the GUI, and what work is planned to be done in the future. On a lower level, Nekotekina reworked the entire PPU LLVM recompiler to greatly increase compatibility of it. kd-11 fixed various graphical issues affecting hundreds of games, most notably broken shadows and depth of field in various titles. Moreover kd-11 also enabled Vulkan on Linux and pushed support for it to the master branch. Nekotekina and Numan also worked on .sprx relocations in general and in the LLVM recompiler which in simple terms is a compatibility and performance improvement.
The talented developer Jarves is working on a very important core improvement. It is not yet finished or merged to the master branch, but as work on this has been going on in public, and a lot of people have already tested it and submitted results this report will cover the work anyway. Jarves is working on “LLE gcm” which is huge and here is why:
LLE stands for Low Level Emulation. In RPCS3 this means that a PS3 operating system module is run directly as it is via lower level emulation methods. This is great for compatibility because it doesn’t involve much guesswork and is very accurate. As far as the games are concerned, they are working with the exact same operating system methods with the exact same implementations here as on a real PS3.
gcm or “graphics command management” is the part of the PS3 operating system responsible for creating and managing various graphics commands, including everything from how to set up vsync to graphics memory allocation.
LLE gcm is huge. A very core part of emulating games is now done exactly the same as on on a real PS3, and the result is amazing. For example Red Dead Redemption no longer hangs almost instantly. The Last of Us doesn’t crash immediately and actually shows a loading screen and goes through a lot of initialisation successfully. Persona 5 no longer randomly crashes. Various high end games that did nothing before are now starting to boot, for example the Yakuza series where Yakuza 3 went from nothing to ingame, and Yakuza 4 and 5 went from nothing to loading screens. One could go on and on about what a substantial compatibility improvement LLE gcm is (and I will). Now combine this with the greatly improved LLVM recompiler and over 40 commits of graphics fixes and you have a month of quite insane progress. This would be a great segway into looking at improved games and such, but Jarves did one other important contribution that deserves its own section.
59.94 hz or “the dumbest bug of all time”
Koei Tecmo including subsidiaries like Gust, Team Ninja, and Omega Force have developed several games in several popular series like Warriors, Atelier, Dead or Alive, and so on. They all share the same engine and every game in this engine since 2014 failed to work at all in RPCS3. In fact, even going back to RPCS3 versions from 2015 these games fail in exactly the same way, which is interesting because there basically isn’t an area of the emulator that hasn’t been touched since then (you see where this is going). Various people had looked at it and (incorrectly) concluded that perhaps some colossal area like cellSpurs or gcm were failing. As even Red Dead Redemption started to work before the weeb games that I (ssshadow) am interested in, like Atelier Escha & Logy, this engine had reached a meme status for RPCS3 developers. A week ago two things happened:
ssshadow looked at an older game in the same engine that did work (Deception IV) and found that cellVideoOutGetDeviceInfo() was the last function call before cellVideoOutConfigure() and various gcm functions with various parameters, that is the last function call before initialising graphics output is cellVideoOutGetDeviceInfo(). But in Atelier Escha & Logy (and every other game in this engine since 2014) cellVideoOutGetDeviceInfo() was the last function call before the game got into an endless loop of doing nothing useful at all, or in one case throwing an exception. cellVideoOutConfigure() was not called! Let’s look at the code:
“Use standard dummy values for now.”. Could it be?
Jarves took a closer look and ran one of the games in the same engine that he had. Shortly thereafter Atelier Escha & Logy, and really 20 other games too, booted for the first time ever in RPCS3, and Atelier here even went ingame. cellVideoOutGetDeviceInfo() was indeed to blame. In particular a “standard dummy value”. This is the commit that fixes everything:
Yes. Every game in this engine since 2014 explicitly and literally refuses to boot and enters a dead code path of doing nothing or even throwing an exception if the “display” doesn’t report supporting 59.94 hz. One can disassemble the binary, and Jarves did, and see that is how they programmed it. If refresh rate is not 59.94 hz, do not boot. No one will ever know why they did this.
IndexJump to:
Games
Linux
Ultrawide mods
The new Qt interface
Commits
Nekotekina
kd-11
Other Contributors
Conclusion
Closing Words
In conclusion, the new Qt GUI by flash-fire, Megamouse, hcorion and DAGINATSUKO, improved LLVM recompiler by Nekotekina, graphics fixes by kd-11, and work on LLE gcm by Jarves has lead to some monumental RPCS3 progress and a very long progress report.
If you want to help out Nekotekina and kd-11 and make RPCS3 progress even faster you can check out the Patreon page...
This progress report was written by ssshadow with contributions from twdarkeh, Ani, MarioSonic2987, and Talkashie.
https://rpcs3.net/blog/2017/07/03/progress-report-june-2017
Another month another colossal RPCS3 progress record. But first things first, take a look at this beautiful introduction to RPCS3 video that reznoire made:
Moving on to some statistics the numbers alone this month are quite impressive, and that is not even counting games that improved from emulation improvements that are still work in progress.
Looking at the GitHub statistics 16 authors have pushed 140 commits to the master branch. Here 353 files have changed and there have been 19,729 additions and 17,660 deletions. These numbers are much larger than normal, and several exciting changes are behind it. This month RPCS3 transitioned away from the previous GUI toolkit Wx to Qt which in turn also lead to several user facing interface improvements. This separate report goes into significantly greater detail regarding what work has already been done on the GUI, and what work is planned to be done in the future. On a lower level, Nekotekina reworked the entire PPU LLVM recompiler to greatly increase compatibility of it. kd-11 fixed various graphical issues affecting hundreds of games, most notably broken shadows and depth of field in various titles. Moreover kd-11 also enabled Vulkan on Linux and pushed support for it to the master branch. Nekotekina and Numan also worked on .sprx relocations in general and in the LLVM recompiler which in simple terms is a compatibility and performance improvement.
The talented developer Jarves is working on a very important core improvement. It is not yet finished or merged to the master branch, but as work on this has been going on in public, and a lot of people have already tested it and submitted results this report will cover the work anyway. Jarves is working on “LLE gcm” which is huge and here is why:
LLE stands for Low Level Emulation. In RPCS3 this means that a PS3 operating system module is run directly as it is via lower level emulation methods. This is great for compatibility because it doesn’t involve much guesswork and is very accurate. As far as the games are concerned, they are working with the exact same operating system methods with the exact same implementations here as on a real PS3.
gcm or “graphics command management” is the part of the PS3 operating system responsible for creating and managing various graphics commands, including everything from how to set up vsync to graphics memory allocation.
LLE gcm is huge. A very core part of emulating games is now done exactly the same as on on a real PS3, and the result is amazing. For example Red Dead Redemption no longer hangs almost instantly. The Last of Us doesn’t crash immediately and actually shows a loading screen and goes through a lot of initialisation successfully. Persona 5 no longer randomly crashes. Various high end games that did nothing before are now starting to boot, for example the Yakuza series where Yakuza 3 went from nothing to ingame, and Yakuza 4 and 5 went from nothing to loading screens. One could go on and on about what a substantial compatibility improvement LLE gcm is (and I will). Now combine this with the greatly improved LLVM recompiler and over 40 commits of graphics fixes and you have a month of quite insane progress. This would be a great segway into looking at improved games and such, but Jarves did one other important contribution that deserves its own section.
59.94 hz or “the dumbest bug of all time”
Koei Tecmo including subsidiaries like Gust, Team Ninja, and Omega Force have developed several games in several popular series like Warriors, Atelier, Dead or Alive, and so on. They all share the same engine and every game in this engine since 2014 failed to work at all in RPCS3. In fact, even going back to RPCS3 versions from 2015 these games fail in exactly the same way, which is interesting because there basically isn’t an area of the emulator that hasn’t been touched since then (you see where this is going). Various people had looked at it and (incorrectly) concluded that perhaps some colossal area like cellSpurs or gcm were failing. As even Red Dead Redemption started to work before the weeb games that I (ssshadow) am interested in, like Atelier Escha & Logy, this engine had reached a meme status for RPCS3 developers. A week ago two things happened:
ssshadow looked at an older game in the same engine that did work (Deception IV) and found that cellVideoOutGetDeviceInfo() was the last function call before cellVideoOutConfigure() and various gcm functions with various parameters, that is the last function call before initialising graphics output is cellVideoOutGetDeviceInfo(). But in Atelier Escha & Logy (and every other game in this engine since 2014) cellVideoOutGetDeviceInfo() was the last function call before the game got into an endless loop of doing nothing useful at all, or in one case throwing an exception. cellVideoOutConfigure() was not called! Let’s look at the code:
“Use standard dummy values for now.”. Could it be?
Jarves took a closer look and ran one of the games in the same engine that he had. Shortly thereafter Atelier Escha & Logy, and really 20 other games too, booted for the first time ever in RPCS3, and Atelier here even went ingame. cellVideoOutGetDeviceInfo() was indeed to blame. In particular a “standard dummy value”. This is the commit that fixes everything:
Yes. Every game in this engine since 2014 explicitly and literally refuses to boot and enters a dead code path of doing nothing or even throwing an exception if the “display” doesn’t report supporting 59.94 hz. One can disassemble the binary, and Jarves did, and see that is how they programmed it. If refresh rate is not 59.94 hz, do not boot. No one will ever know why they did this.
IndexJump to:
Games
Linux
Ultrawide mods
The new Qt interface
Commits
Nekotekina
kd-11
Other Contributors
Conclusion
Closing Words
In conclusion, the new Qt GUI by flash-fire, Megamouse, hcorion and DAGINATSUKO, improved LLVM recompiler by Nekotekina, graphics fixes by kd-11, and work on LLE gcm by Jarves has lead to some monumental RPCS3 progress and a very long progress report.
If you want to help out Nekotekina and kd-11 and make RPCS3 progress even faster you can check out the Patreon page...
This progress report was written by ssshadow with contributions from twdarkeh, Ani, MarioSonic2987, and Talkashie.
