Modding Oblivion on Linux

As mentioned in my last post, I recently moved to Linux. For the most part this has been going well and most problems I ran into were expected. Many of my favorite games are native, or work excellent in Proton/Wine. A lot of my programs work, too, and I found alternatives for most of the rest.

And then there is Oblivion.

Truth be told, I avoided installing Oblivion until the last moment. I did about everything else before I started doing that. A bit late spring cleaning. Sorting all my bookmarks of the last 10 years. Painting a kitchen cabinet. Sorting my Dropbox of the last 10 years. But finally, there was almost nothing else left to try to get to work.
As for the reason why I avoided it, I guess I was kinda scared, because the more I liked my new system, the more I feared that my favorite game just wouldn't work like I wanted it.

Now please keep in mind I am a total noob and don't know anything about the inner workings of Linux. Setting up games has pretty much been a case of installing Steam and Lutris and clicking "play". Between people telling me that my PC will blow up and I won't be able to play anything on Linux and instructions telling me that it's easy to fix bug x, just edit wine and compile it yourself, I wanted to write down what worked for me.

Kat's blog has been immensely helpful (and I probably drove them mad with all my dumb questions).

I'm using Manjaro with XFCE and a Nvidia card.

The Base Game

Steam

Getting Oblivion to run by itself is easy enough. It works well with Proton. I had to use the launch command PROTON_USE_WINED3D11=1 %command% (might have been a problem with Vulkan on my end) and it had way better performance with Proton 4.11-13 than 5.0-9, which at the time of this post is the most recent one. I didn't want to use the Steam version, though, since from what I've read OBSE is a huge pain in the butt with it, since you need to edit game files directly. OBSE works now.

If you need to make this work, check out Kat's blog or my own post about modding Oblivion on Steam.

Wine

For the GOG version, I followed Kat's guide, which also worked quite well. I installed the Game, quickly tested it, installed OBSE & the GOG loader normally and instead of OblivionLauncher.exe or Oblivion.exe, I just put obse_loader.exe in all the scripts. The game launched and all was well.

I'm currently using Lutris instead of system wine, because I had some hiccups after updating wine (and with Manjaro being a rolling release, it updates constantly) and because it is very easy to handle additional apps/environmental variables (libstrangle, vkBasalt, etc) with Lutris.

Lutris

Lutris is a platform similar to PlayOnLinux that allows one-click installs for a lot of games, often with scripts that apply settings that are known to work with that particular game. It has an interface that allows to easily change settings or create shortcuts for installed games and it already comes with DXVK.

I used Lutris with the downloaded offline backup installer from GOG - not the top one that comes with Galaxy. Just point the script to the downloaded exe that's in the same directory as the 2 bin files that came with it. I don't think the Lutris script does anything special other than just installing the game.

After installing libstrangle, Lutris can cap the game's FPS. This is the first time a FPS cap has worked for me in Oblivion!

It is also super easy to add custom versions of wine to single games, which is what I did later.

All Versions

• Intro videos cannot be skipped. You can disable them in the ini by setting

  SIntroSequence=

• Some audio libraries were missing for me. On my old PC, lib32-gst-plugins-good was missing for me. On the new one, I was missing lib32-gst-plugins-base. If Oblivion launches, but freezes/crashes after a moment, I would suggest checking all gst-plugins, especially the 32 bit ones that start with lib32.

Tools that work and hiccups

Steam

Didn't try. Tried Wrye Bash, the CS itself now and Tes4Edit.

Wine

While I could launch tools on their own by just running them in the same wineprefix as Oblivion, I like the toolbar approach and just launch them via Wrye Bash instead. After all, that's what I did on Windows already. I launch Wrye Bash and used the app shortcuts to launch additional tools & the game. I made notes about a workaround needed to launch some of the tools on the Wrye Bash post.

A note: Whenever available, I chose the manual install for programs, even if they offer an installer, i.e. just downloading the archive and dropping the program into a location of my liking, for example drive_c/tools.

Tools I got to work so far:

• The Construction Set works fine, but it has some hiccups:
  Sometimes using Ctrl-V on the script window replaced the whole script, even if nothing was marked, so it was impossible to just paste in a small snippet into existing code. I usually copy my scripts into a useful editor (i use neovim with Obli-Vim syntax highlighting), do my changes, and copy it back.
  I cannot use a key on my keyboard to quickly jump to a script name in the open dialogue, i.e. typing "mania" in the Tree section will jump to the ManiaBush01 tree, but pressing any letter in the script open dialogue will only select the first script.
  I can also not select anything in most other dropdown menus by pressing more than one letter, i.e. if I want to select an owner for a container, I can press G to jump to the first NPC starting with G, but I cannot type GU to find Gundalas, it would jump to the first NPC starting with U.
• I am using a floating window manager, if you use a tiling window manager, you might have to configure it to your liking, as the CS consists of a lot of windows.
• I am not using the CSE. I know I know, it's bad to not use it. I'm too lazy. I can only once more point to Kat's blog, as they describe requirements and workarounds to get it to work.
• xEdit works fine as far as I can tell. Kat gave me the tip to set mode to Windows XP in winecfg if Drag & Drop doesn't work. You can set this for the Tes4Edit.exe only.
• BOSS works without a problem. I always use just the BOSS.exe, not the GUI, and edit the userlist myself without fiddling with the UI. If launching from Wrye Bash, it needs to be bound to another tool's icon. Launching it from the BOSS icon made Wrye Bash freeze for me.
• LOOT seemed to work as well, but the version that worked for me was v0.15.1 from Dec 2019. The latest release from Aug 31 2020 did not launch and I have not tried any more recent releases, if they exist. I did run the installer for this one because it installs the required MSVC 2017 x86 redistributable. I installed it by selecting "Run EXE inside wine prefix" in Lutris, then added the installed LOOT.exe to my Wrye Bash toolbar. Please note that the installer needs the wineprefix to be at least Windows 7.
  I don't know if it is really necessary to install the MSVC 2017 x86 redistributable in Lutris. I tried the archive version, but it didn't launch because it was version v0.16.0, which I didn't know was the cause at this point.
• Nifskope - Seems to work. I had to type the path to the textures folder manually, as the selector didn't work for me, when adding them to the settings. It did respond a bit of laggy to my input in other cases as well as sometimes having issues with updating what should be rendered, but for a quick change to a texture path or something, it should work.
• Tes4LodGen - Works fine.
• OBMM - I only use OBMM to extract and pack bsas. Only used it with Lutris so far. It crashes on exit sometimes, but worked fine otherwise. In my non-Lutris wine prefix, something was missing that made it impossible to launch OBMM.

Lutris

Tools work just as well, but are harder and easier to start at the same time, since Lutris runs games with its own runners. Using Lutris' option of launching a program in the wineprefix also didn't work too well for some, which could or could not have been fixed by now. It's worth a try I guess.

Pointing Lutris to Wrye Bash (Standalone Executable) instead of Oblivion.exe and using Wrye Bash as a toolbar to start everything else works fine.
So launching one app with Lutris is easy, switching them around is a bit annoying, hence the toolbar approach.

Wrye Bash: A bit annoying, but works mostly

• This is too much text, it'll just get its own post.

ENB: The Worst

Another thing I tried getting to work was an ENB. This was not fun (and in the end, not really successful).

I downloaded the files from enbdev, I dropped the d3d9.dll into the Oblivion folder and I went into my Oblivion wine prefix' winecfg and set the override for d3d9 to "native, builtin" just for Oblivion.exe. That's basically the same for all versions, just the wineprefixes are different.

• In that wineprefix I also ran winetricks d3dx9 which installs a bunch of dlls needed for this to work
• If Lutris is running with DXVK, and you don't set the override, as long as the dll is in Oblivion folder, it will load ENB. This made DXVK stop working for me, though, best to check if both work with turning on DXVK HUD.

The ENB loads and works and applies its effects, but the performance is abysmal. Not even low fps, but it seems like everything takes ages to load. The first loading screen takes, unmodded, 30 seconds on a ssd and whenever I cross a cell boundary ingame, my frames drop from 120 to 5 as stuff is loaded.

I haven't been able to find a workaround for that yet. Most posts I found (and there aren't that many) are about newer games which combine enboost and enb and offer solutions that can't be applied to Oblivion.

I tried disabling all effects - no luck. Changing random settings around (in enbseries.ini) - no luck. As long as nothing is loading in, performance is excellent, but as soon as anything is loading, it drops. Cell boundaries, menus, spell effects, doesn't matter what.

I deemed it unplayable and dropped it.

Instead: Oblivion Reloaded

I did have some weird happenings when I threw Oblivion Reloaded. I do still use 6.5 though, since I use a mod that isn't compatible with 7+. I did try some 7.x versions, as well as some 8.x versions and had some issues with any of those. I am too lazy to try out 9.x, but I don't think anything changed about my issues.

• One bug I encountered with every Oblivion Reloaded version I tried is shaders and geometry being off in first person mode. My enchanted sword looked like plastic and I had some weird objects sticking out when I readied my weapon. When wearing gloves - but no weapon - I had kind of a second hand model stuck over my hands. Using OR's camera mode would prevent that issue, as there technically is no first person camera anymore, it's fake.
  This is an issue with dxvk, which windows users can also experience, if they use dxvk. There is no known workaround.
• It's hard for me to tell which bugs in later OR versions are due to Linux, or due to OR being OR or. I had volumetric fog bug out, landscape shaders breaking water reflections, hair turning blue and just now 8.4 broke my grass in a way that didn't even get fixed when uninstalling, I had to reset my ini. Shadows aren't worth the trouble for me to keep trying.
  Good news: I believe all my experienced bugs were not Linux related, just OR things™.
• If you find a version that works for you, hold on to it, keep it, store it, throw it on your cloud or a pen drive, as old versions keep getting taken down.
• After loading the game with Oblivion Reloaded I had super bad performance that went away after a few restarts. I believe it was DXVK compiling OR shaders, but don't quote me on that.
• To be able to compile custom shaders with Oblivion Reloaded, I needed to use winetricks d3dcompiler_43.

Another Thing: vkBasalt

This is a tool similar to Reshade on Windows. The downside is that - unlike ENB and Oblivion Reloaded - it will also affect the inventory/interface which can look pretty bad. I think there are other alternatives to Reshade, but I use this one because it's pretty easy to add to Steam or Lutris; just drop in a config file and set a variable.

Other alternatives might be closer to true reshade, with better shader compatibility.

I found vkBasalt and a pack of Reshade shaders in the AUR, so I can't tell you how to install those on other distros.

Updates:
2021-03-06: Fixed some things, added vkBasalt
2021-07-25: Just a bit cleanup.