Page 3 of 3

Re: Development Blog

Posted: Sat Apr 22, 2023 8:39 pm
by CrabHead
Before i go enact plan "Sleep", i want to share a minor bug in the menu's. When scrolling through options in the main menu, the text below explaning what each option does, glitches by having a duplicate underneath the original location. It doesn't happen in any other menu, like drivers profile, crash nav etc. I might've explained it poorly so i include a video to show.

Re: Reburn 3: Development Blog

Posted: Fri May 05, 2023 11:10 am
by MattKC
Haven't had much time to work on this lately due to some personal life stuff I've been sorting out, but in the mean time I have been thinking about a kind of conspiracy theory I want to discuss.

Cxbx started pretty early in the Xbox's life. From what I've read, it was at the proof of concept stage, working with sample applications from the XDK, within only a year of the Xbox's lifespan in late 2002, and could play commercial games not long afterward (well, a game, specifically Turok Evolution).

Microsoft must have known about this. They were well aware when "bunnie" dumped the secret ROM and published it on his website, so it's not a stretch to think they would have seen this too. Guess what else happened in 2002?

LTCG was implemented in Microsoft Visual C++. (article)

LTCG, as you're probably aware from my earlier posts, is one of the biggest roadblocks for Cxbx, exacerbated by its use in a huge amount of Xbox games. It's the thing that takes the concept of inlining and turns it up to 11. As you can read in that article above, functions optimized by LTCG are not just frequently aggressively inlined, but even the functions that aren't are still heavily optimized to the point that they don't end up following standard calling conventions anymore. Instead, they pass as many parameters as possible via registers (similar to fastcalls, but even more so), which I've noticed a lot in my analysis of B3 code.

This is a nightmare for reverse engineering, and especially for injecting, because the way functions are called can no longer be easily replicated with normal C++ code. To match what LTCG did in a lot of these cases, I will probably need to write some glue in raw x86 assembly to stitch their calls to my functions. It's a very manual and complex process that otherwise wouldn't need to exist, and it effectively prevents Cxbx from making clean injections/replacements of Xbox system calls; preventing it from being an ideal solution.

Am I suggesting LTCG was added because Microsoft discovered Cxbx? No, almost certainly not. The timing is too fast, you couldn't implement a major linker feature like that in such a short amount of time. Besides, other compilers (like the Intel C++ compiler) already had this feature, so it would be a fairly natural progression for them to make.

But I do wonder if there was some relation to the idea. Perhaps Microsoft realized injection could be used to run games on Windows while the Xbox was still being developed in 1999-2001, and set into motion plans to get LTCG implemented as soon as possible in their linker to circumvent that (also standardizing the XDK around statically linking all system libraries rather than dynamically linking, which would have made injecting them dead simple). There are in fact slight differences between the Win32 API and Xbox API (e.g. D3DSHADE_GOURAUD being 0x2 on Windows but 0x1D01 on Xbox) that it's hard to believe exist for any other reason than to prevent direct binary compatibility with Windows.

The prevalence of LTCG throughout the Xbox library certainly suggests Microsoft strongly recommended developers to use it. Up until now, I assumed this was to squeeze as much performance out of the hardware as they could (keeping the console competitive for as long as possible), but maybe it was actually a form of implicit DRM too. Maybe Microsoft knew this would break approaches like Cxbx...

Re: Reburn 3: Development Blog

Posted: Fri Aug 18, 2023 5:37 pm
by CrabHead
Can't believe Matt cheated Burnout 3 in favor of lego island i will never forgive him!!!

Re: Reburn 3: Development Blog

Posted: Mon Aug 21, 2023 11:00 pm
by MattKC
Lol sorry for going MIA, it's true that the LEGO Island decomp has taken a bit more of my time, but my plan has always been to alternate between these two projects, and the only reason I haven't yet is because of my silly 2-month-long move. Don't worry, I'll be back soon!

Re: Reburn 3: Development Blog

Posted: Fri Aug 25, 2023 12:54 pm
by CrabHead
good to know!

Re: Reburn 3: Development Blog

Posted: Fri Dec 08, 2023 12:17 am
by MattKC
Work has started back up on Reburn 3! It's taken longer than I'd hoped, but we're officially making progress once again!

To be honest, part of the reason I was putting it off was because I felt a little unsure about the direction to take this long-term. As I've alluded in earlier updates, I'm currently working with two builds of the game: the Xbox build, which can be made to run on PC with the help of Cxbx-Reloaded, and the prototype PS2 build, which I have a large amount of documentation for.

But the Xbox version has a big problem from a reverse engineering standpoint. It uses heavy LTCG (Link-Time Code Generation) optimizations that make it immensely more difficult to work with and modify. What could have been a simple injecting/hooking process now requires significantly more effort.

It's been enough for me to consider just saying "fuck it" and decompiling the documented PS2 prototype instead. But then we'll be waiting ages (potentially years) before we actually see any results.

So it's tricky. Ultimately, I want the portable codebase that we would get from a full decompilation, but we have something today that's probably 85% of the way there to a usable Windows port. It's hard to know which approach is worth prioritizing; which one is more worth my time.

What I've decided is somewhere in between. I'm still going to build off the Xbox version - still using the PS2 proto as a reference - but I'm not just going to correct the handful of Xbox push buffer calls so it runs well on Windows.

Instead, I'm going to try my hand at rewriting the graphics system, so it could theoretically run well on anything. I see this as a reasonable compromise - we don't have to decompile the whole game before we get results, but we're still progressing towards what could be a full decomp one day, rather than just tweaking some low-level Xbox stuff.

So how is this going to work? Well, hopefully, this will actually be easier than it sounds. Burnout, as I've mentioned earlier, uses RenderWare, whose primary selling point (apart from making the PS2 easier to develop for) was cross-platform compatibility.

In theory, detaching the game from its own Xbox version of RenderWare and linking it up with the Windows version of RenderWare - and smoothing over platform differences as necessary - should work. Unfortunately, this is complicated by the fact that LTCG has inlined much of the RenderWare and D3D8 code; it can often be hard to know exactly where one function begins and another ends. But hey, what's reverse engineering without some new puzzles and challenges to be solved?

If this works out, it should then be pretty trivial to switch it over to librw - an open source reimplementation of RenderWare that supports more modern platforms like Linux, macOS, and the Nintendo Switch, and also shields us from having to distribute any confidential RenderWare code. Once again, this will bring us a significant amount of the way towards a portable/decompiled codebase.

So in the interest of setting small achievably goals for myself, my first is simple: try to get the initial "loading" screen to appear entirely natively through Windows RenderWare.

Screenshot_20231207_160613.png

This screen is super simple, but is still assembled and rendered in realtime by the game (you can see in another thread where I was able to edit the text). It's also completely isolated from the main game loop, so it should be relatively easy to get working without having to worry about the rest of the game.

RenderWare proper supported D3D8, D3D9, and OpenGL. If we can get this loading screen to display in all three, it'll be the first time Burnout 3 has ever rendered natively in OpenGL. I don't know about you, but that sounds pretty interesting to me.

Stay tuned, I'll keep you guys posted!

P.S. I've also updated the project's README page to better reflect Reburn's updated goals and make it easier for newcomers to understand what we're trying to do.

Re: Reburn 3: Development Blog

Posted: Sun Dec 10, 2023 9:03 pm
by CrabHead
Yay its not dead

Re: Reburn 3: Development Blog

Posted: Mon Dec 11, 2023 6:12 pm
by Dogeyboy
Eyy some progress Clap

Re: Reburn 3: Development Blog

Posted: Mon Jan 08, 2024 12:14 pm
by DjArcas
MattKC wrote: Fri Dec 08, 2023 12:17 am Work has started back up on Reburn 3! It's taken longer than I'd hoped, but we're officially making progress once again!

To be honest, part of the reason I was putting it off was because I felt a little unsure about the direction to take this long-term. As I've alluded in earlier updates, I'm currently working with two builds of the game: the Xbox build, which can be made to run on PC with the help of Cxbx-Reloaded, and the prototype PS2 build, which I have a large amount of documentation for.

If it's any motivation whatsoever; I was the audio programmer on Burnouts 2/3/4/360/Legends/Dominator, and I'm following this process with fascination. I'd personally go with the Xbox version, it's a lot better *and* the PS2 was hard enough to work with when Sony was helping us.

Keep it up <3

Re: Reburn 3: Development Blog

Posted: Wed Jan 10, 2024 7:44 am
by MattKC
DjArcas wrote: Mon Jan 08, 2024 12:14 pm If it's any motivation whatsoever; I was the audio programmer on Burnouts 2/3/4/360/Legends/Dominator, and I'm following this process with fascination. I'd personally go with the Xbox version, it's a lot better *and* the PS2 was hard enough to work with when Sony was helping us.

Keep it up <3
No way, welcome to the thread! It's an honor to have you here reading!

Progress has been a little slower than I would have liked, but I hope to kick it into full speed (pun sort of intended?) very soon. Thanks for the motivational push!

Re: Reburn 3: Development Blog

Posted: Tue Apr 09, 2024 8:55 am
by berkekurdu
Is there any progress?

Re: Reburn 3: Development Blog

Posted: Thu Jun 13, 2024 6:03 pm
by Dan
MattKC wrote: Fri Dec 08, 2023 12:17 am As I've alluded in earlier updates, I'm currently working with two builds of the game: the Xbox build, which can be made to run on PC with the help of Cxbx-Reloaded, and the prototype PS2 build, which I have a large amount of documentation for.
...
It's been enough for me to consider just saying "fuck it" and decompiling the documented PS2 prototype instead. But then we'll be waiting ages (potentially years) before we actually see any results.
Could you explain what's particularly beneficial about the PS2 prototype for this project? You hinted considering re-doing the project to base off of it (instead of the final released version).
Does it have debug symbols that the final doesn't? Is it due to functionality that was in the prototype that didn't make it to final?

Just wanted to ask, as it piqued my interest. Of course a prototype with debug symbols like DWARF could be greatly helpful, but I didn't think that was the case.

Re: Reburn 3: Development Blog

Posted: Fri Jun 28, 2024 2:13 pm
by CrabHead
Dan wrote: Thu Jun 13, 2024 6:03 pm Could you explain what's particularly beneficial about the PS2 prototype for this project? You hinted considering re-doing the project to base off of it (instead of the final released version).
Does it have debug symbols that the final doesn't? Is it due to functionality that was in the prototype that didn't make it to final?

Just wanted to ask, as it piqued my interest. Of course a prototype with debug symbols like DWARF could be greatly helpful, but I didn't think that was the case.
IIRC the june 8 prototype has debug symbols