OSBuilder8: the development process Announcements, Research

2014-03-25

Usually I don’t post any details of projects being actively developed. But this time I decided to do that just to let you know that I am still alive. So, it all started with a simple idea: what about transferring OSBuilder’s rom building experience to Windows Phone 8? Huh, important consideration here: there are not that many devices that can take an advantage of custom ROM. Huawei W1, 8X/8S (theoretically), Ativ S are on the top of the list. Anyway, since I have engineering Ativ S with somewhat like pre-RTM ROM on board, I started development of the tool which main idea was to update my device to GDR3 or so.

Devices

Package structure considerations

Basic OSBuilder’s folder structure is left intact. As for package paths. OSBuilder uses paths like these: %OwnerType%\\%Owner%\\RES*\*\\LANG*\*\\%Identity.Component%\\%Identity.SubComponent%

i.e. path for Microsoft.MainOS.Production package (ru-ru) will look like: SYS\\Microsoft\\LANG*ru-ru\\MainOS\\Production

Package internals

After all, I am yet to find anything better than keep file names the way they are in CABs (%magicnumber%*%fewlettersofname%.%extension%). Why? Because some files in the package might have the same name but belong to different folders on real device. Increasing folder complexity is a nasty idea (don’t forget about Windows folder path limit - not saying anything about overcoming this limit).

Registry

For a better control over registry building process, I’ve made the whole new registry parser. What we have: .reg files - easy to add. Simply add or replace content of registry value in the given key. .rga files - usually they are about adding something to REG_MULTI_SZ (multi-string values). Order is usually not important here, but I tried to keep it as close to stock as possible. While adding content of .reg files is basically easy, the biggest problem is to keep original order.

Package order

Package order can be found in \\Windows\\ImageUpdate\\UpdateHistory.xml file. But don’t head over to it. Don’t fully trust it. After some experiments I realized that some packages still have to be moved up. While it doesn’t render UpdateHistory.xml file useless, it might break real building order, might lead to incorrect registry values, non-relevant policies etc… Trying to be careful here.

Policies

That’s probably the most time consuming part of the whole process. We have to rebuild policy database (stored in registry) in order to keep it in sync with .policy.xml files. Policy database contains all rules like User Ivan can open folder C:\\FunnyCats, Administrators can add new kitties to folder C:\\FunnyCats. Without correct database, it is impossible to boot Windows Phone - it will simply fail in the beginning of the process (not really beginning, but still). Imagine: you have file like this:

<?xml version="1.0" encoding="utf-8"?>
<PhoneSecurityPolicy xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" HashType="Sha256" PackageID="Microsoft.BaseOS.IpOverUsb" xmlns="urn:Microsoft.WindowsPhone/PhoneSecurityPolicyInternal.v8.00">
  <Capabilities>
    <Capability xsi:type="PrivateResources" ElementID="08826C28AC9B35E6DC9BEEE6CAB066A603CFBA91C157DD765F1EE7C475D0548D" AttributeHash="C4F906E3940D86900E1EDAE6241A799DF3918136DCB3686A3EBC66A71F9F12E1" Id="ID_CAP_PRIV_IPOVERUSB" SvcCapSID="S-1-5-80-3281897998-4027600974-1982452685-2998814832-604313930" FriendlyName="IPOVERUSB private capability" Visibility="Private">
      <CapabilityRules>
        <Rules>
          <SDRegValue ElementID="B5320ADB6B754E2CCC033A46DD49B63FE82C64ED8FB2694FCFC277FE10C3DDF7" DACL="(A;;0x111FFFFF;;;S-1-5-80-3281897998-4027600974-1982452685-2998814832-604313930)" Flags="2147483652" Path="HKEY_LOCAL_MACHINE\SYSTEM\CONTROLSET001\CONTROL\NOTIFICATIONS\41C61D22A3BC8075" Type="WNF" />
        </Rules>
      </CapabilityRules>
    </Capability>
  </Capabilities>
  <Components>
    <Service ElementID="EE71E815BB0E265E208AA56A3B2C4AEE3FFB782C26A41203AA65549F16F01780" AttributeHash="67E67DB70A5BE2BB6E8BCECABD139168A4E95C76A4E43A8E186C0C2527DFBBDF" SID="S-1-5-80-3281897998-4027600974-1982452685-2998814832-604313930" Name="IPOVERUSB" PrivateCapSID="S-1-5-80-3281897998-4027600974-1982452685-2998814832-604313930" Executable="\windows\System32\SvcHost.exe" IsTCB="No" SvcHostGroupName="IPOVERUSBGROUP">
      <RequiredCapabilities>
        <RequiredCapability CapId="S-1-5-21-2702878673-795188819-444038987-1543" />
      </RequiredCapabilities>
    </Service>
  </Components>
</PhoneSecurityPolicy>

“Wow, that’s a lot of text”, you will say. And you’re right. There are a lot of entries like this and they all look similarly. Still, we need to parse all of them and make something useful from it. I turn it to something like that:

[HKEY_LOCAL_MACHINE\\System\\SecurityManager\\Capabilities\\ID_CAP_PRIV_IPOVERUSB\]
"CapabilityFriendlyName"="IPOVERUSB private capability"
"CapabilityType"=dword:00000002
"ServiceCapabilitySID"=hex:01,06,00,00,00,00,00,05,50,00,00,00,0E,CA,9D,C3,4E,50,10,F0,CD,D3,29,76,70,48,BE,B2,4A,19,05,24
"Privileges"=hex:00,00,00,00 "EmbeddedWindowsCapabilitySIDs"=hex:00,00,00,00

[HKEY_LOCAL_MACHINE\\System\\SecurityManager\\Capabilities\\ID_CAP_PRIV_IPOVERUSB\\Microsoft.BaseOS.IpOverUsb\]
"PackagePrivileges"=hex(7):00,00,00,00
"PackageEmbeddedWindowsCapabilitySIDs"=hex(7):00,00,00,00
"SourceID"="ID_CAP\_PRIV_IPOVERUSB@Microsoft.BaseOS.IpOverUsb"

[HKEY_LOCAL_MACHINE\\System\\SecurityManager\\Capabilities\\ID_CAP_PRIV_IPOVERUSB\\Microsoft.BaseOS.IpOverUsb\\PackageRules\]
"B5320ADB6B754E2CCC033A46DD49B63FE82C64ED8FB2694FCFC277FE10C3DDF7"=dword:20000000

[HKEY_LOCAL_MACHINE\\System\\SecurityManager\\ResourceAccessControl\\OpaqueObject\\B5320ADB6B754E2CCC033A46DD49B63FE82C64ED8FB2694FCFC277FE10C3DDF7\\ID_CAP\_PRIV_IPOVERUSB@Microsoft.BaseOS.IpOverUsb\]
"DACL"="(A;;0x111FFFFF;;;S-1-5-80-3281897998-4027600974-1982452685-2998814832-604313930)"
"DefaultSettingsFlag"=dword:80000004

Not much better. But Windows is ready to use it. Basically, there is no documentation “how to do that”, but I do analysis. I do things in the way I think it should be, compare to stock values and alter behaviour a bit. Repeating until 0 differences are found. Looks like that: Policies

NTFS permissions

Based on the policies parsed above, I have to set correct NTFS permissions on the whole folder structure. After rebuilding, of course. Currently I install rebuilt OS right on device. First wiping the whole MainOS partition and Data partitions, then pushing new files, creating symbolic links, then applying permissions and set file/folder owners. Pretty quick process, in fact. Without it, OS doesn’t boot as well.

How does OSBuilder look?

At this point you might wonder how does this OSBuilder looks at all. It is unbelievably attractive. OSBuilder - 1 OSBuilder - 2 If you want to hire me for my designer skills… Oh, no! No queuing.

“How do you test it?”

Well… Just compare to stock. Not a bad idea usually. Also I test it on my Ativ S (from time to time). It was successfully booting older builds, but I’ve broken something and it doesn’t work right now. Not a big deal.

ETA?

No ETAs.