Premature optimization is the root of all evil

October 5, 2016

If npm tries to install into C:\Program Files

After some uprade (VS 2015 update 3 install?) npm started to install packages into C:\Program Files\nodejs\node_modules, thus requirement admin rights and generally wrecking havoc. Previous installation location was

Npm people claim that this should never happen, while other people claim this happens like all the time. Long story short, this can be overridden by placing an .npmrc file in your home folder (C:\users\{username}) that says


Chances are your package store became corrupted (mine did), so run npm update after you change the prefix.

Typescript compiler, where art thou

I was trying to experiment with Typescript and Visual Studio code, and I found that there are at least three versions of TypeScript compiler one might have:

1. Microsoft compiler tsc.exe that sits in C:\Program Files (x86)\Microsoft SDKs\TypeScript\x.y. It gets installed with Visual Studio. Several versions can be installed side by side: I have 1.0, 1.7, and 1.8, but only one of them will be on the path.

2. Compiler that comes with npm “typescript” package: you run “npm install typescript” and then invoke it via “node node_modules/typescript/bin/tsc“.

3. Compiler that comes with npm “tsc” package.

Compiler #1 seems to work, but it is not very good: it does not support –watch option (at least not on Windows), and it does not support wildcards or include/exclude directives in tsconfig.json, so you must name all files in your project explicitly. So, if you use Visual Studio Code->Command Palette->Configure Task Runner->TypeScript - Watch mode, generated task simply fails. It is a little funny to see a Microsoft product to work on Linux, but not on Windows.

Compiler #2 seems to be very, very old, don’t use it.

Compiler #3 is the one that works best, but it needs to be invoked with an arcane command line.

Bottom line: e pluribus unum may be a good thing in the long run, but it is very, very confusing when you are trying to familiarize yourself with a new technology.

August 25, 2016

Shibboleth: Integration issues

Previous: Shibboleth: controlling access to resources

In this section I discuss non-trivial issues that arose when I tried to integrate Shibboleth with my IdP. Since I procrastinated too much, I only remember three issues worth noting:

1. Shibboleth considers it an exception when it receives a SAML response with AuthnFailed error code. You get a nasty page that looks like a bug report. The normal workflow is to have user authenticated and deny or allow access based on user’s attributes. If users is known, but access is denied Shibboleth will return a civilized 401 Unauthorized.

2. Message “URL is malformed” may happen if IdP incorrectly handles the RelayState, e.g. base64-encodes it twice. Shibboleth expects RelayState to be in certain format, and when it finds that RelayState is broken, it will say “URL is malformed” without much explanation in the logs. If you see this message, ensure that RelayState that comes back in SAML response is exactly the same as RelayState that was sent in SAML request.

3. Hosts/DNS names are very important, because Shibboleth uses cookies to keep state, and cookies are attached to hosts, not IP addresses. If your secure site is, but you try to access it via http://localhost/secure, or, it will “almost” work, but Shibboleth cookie will be lost in the redirects, and Shibboleth will land you at

nUnit test cases with dates

nUnit has a great feature of running multiple similar test cases via data-driven approach:

[TestCase("", "")]
[TestCase("q", "q")]
[TestCase("xyz", "zyx")]
public void TestStringReverse(string s, string expectedResult)
    var result = Reverse(s);
    Assert.AreEqual(expectedResult, result);

However, that does not work with dates, since DateTime is not a primitive type and cannot be used in an attribute.

[TestCase(new DateTime(...), new DateTime(...), false)] // Does not work
public void TestIntervalIsGood(DateTime from, DateTime to, bool isGood)

The solution is to supply test data in runtime, using [ValueSource] attribute. This is more code, but it works.

        // warning: not tested
        public class Interval
            public DateTime From;
            public DateTime To;

        public class TestCase
            public int Id;
            public DateTime From;
            public DateTime To;
            public bool IsGood;

            // We need this override, so test cases are shown well in nUnit. Otherwise they all look like MyTestClass+TestCase
            // We could show actual dates, but it is kind of long, so I opted for a numeric ID
            public override string ToString()
                return "TestCase " + Id;

        private static TestCase T(int id, DateTime from, DateTime to, bool isGood)
            return new TestCase { Id = id, From = from, To = to, IsGood = isGood };

        private static readonly DateTime Before = ...;
        private static readonly DateTime Inside = ...;
        private static readonly DateTime After = ...;

        private static readonly TestCase[] TestCases = 
            T(1, Before, Before, false),
            T(2, Before, Inside, true),
            T(3, Before, After, true)

        public void TestIntervalIsGood([ValueSource(nameof(TestCases))] TestCase testCase)
            bool isGood = IsIntervalGood(testCase.From, testCase.To);
            Assert.AreEqual(testCase.IsGood, isGood);

August 10, 2016

Amusing Cryptography on Windows

Amusing fact #1: .NET framework does not have built-in class to load RSA private key from PKCS#1 (PEM) representation.
I borrowed this:

Amusing fact #2: creating an RSA key requires file system access. To the user profile. If you are running under an ASP.NET app pool user that has no profile, you get this:

System.Security.Cryptography.CryptographicException: The system cannot find the file specified.

   at System.Security.Cryptography.CryptographicException.ThrowCryptographicException(Int32 hr)
   at System.Security.Cryptography.Utils._CreateCSP(CspParameters param, Boolean randomKeyContainer, SafeProvHandle& hProv)
   at System.Security.Cryptography.Utils.CreateProvHandle(CspParameters parameters, Boolean randomKeyContainer)
   at System.Security.Cryptography.Utils.GetKeyPairHelper(CspAlgorithmType keyType, CspParameters parameters, Boolean randomKeyContainer, Int32 dwKeySize, SafeProvHandle& safeProvHandle, SafeKeyHandle& safeKeyHandle)
   at System.Security.Cryptography.RSACryptoServiceProvider.GetKeyPair()
   at {my method}

To fix, one must either create the user profile, or use CspProviderFlags.UseMachineKeyStore when creating RSACryptoServiceProvider. If you choose the latter, make sure the user has write access to C:\ProgramData\Microsoft\Crypto\RSA\MachineKeys. I believe %AllUsersProfile%\Microsoft\Crypto\RSA\MachineKeys should be used to accommodate customized or non-English systems.

If you use machine storage, but the user does not have write access to the above folder, you’ll get
System.Security.Cryptography.CryptographicException: Access denied.

I am not certain which exact calls raises the exception: first Microsoft function on the call stack is GetKeyPair(), but I am not calling it: the actual culprit must be inlined. I suspect it is either new RSACryptoServiceProvider(CspParameters) or RSACryptoServiceProvider.ImportParameters(RsaParameterTraits), I did not have time to investigate which one.

July 29, 2016

Shibboleth: controlling access to resources

Previous: Shibboleth installation and configuration.
Next: Shibboleth: Integration issues.

SAML attributes

Shibboleth allows or denies access to a directory based on the attributes of the user it receives from the Identity Provider. If Identity Provider cannot authenticate the user, Shibboleth will show an ugly looking exception page. If Identity Provider can authenticate the user, Shibboleth will make the access decision based on user’s attributes returned by the IdP. Attributes come back as part of the IdP authentication response and contain a name and a value:

<saml:Attribute Name="mycompany.securityClearance"><saml:AttributeValue>TopSecret</saml:AttributeValue></saml:Attribute>

Attribute names and values are not part of the SAML standard and are IdP specific, although there are some semi-standard attribute names on the Internet.

Mapping SAML attribute names to Shibboleth attribute IDs

In order for Shibboleth to consider an attribute, its name must be mapped to an internal id using attribute-map.xml file typically located at C:\opt\shibboleth-sp\etc\shibboleth\attribute-map.xml. Attributes that are not mapped cannot participate in the authorization decision. For each attribute, add the following line to the attribute-map.xml file:

<Attribute name="nameAsSeenOnTheWire" id="internalShibbolethId" />

Internal Shibboleth ID is arbitrary and does not have to match the name on the wire. E.g., you may have

<Attribute name="myCompany:securityClearance" id="clearance" />

Static access control

Once you have attribute IDs defined, you can control access to protected resources using <RequestMapper> section of shibboleth2.xml configuration file. The following snippet gives access only to the users with Secret or TopSecret clearance:

    <RequestMapper type="Native">
            <Host name="ikrivpc">
              <Path name="secure" authType="shibboleth" requireSession="true">
                    <Rule require="clearance">Secret</Rule>
                    <Rule require="clearance">TopSecret</Rule>

In theory you can have different rules for different paths, but I have not tried that. Full documentation for RequestMap rules can be found in Shibboleth wiki.

Users who are authenticated by not authorized to access the resource will get error 401 (Unauthorized).

Dynamic access control

Being an ISAPI extension, Shibboleth injects mapped attribute values as pseudo-headers into the HTTP request. Dynamic content such as .aspx files can access those values and make finer-grained authorization decisions based on that, or use the attributes in any other way the see fit. For attribute ID fooBar corresponding header name is SHIB_FOOBAR (capitalized). Note, that these are not real headers, you will not see them on the wire. They exist only between the IIS web server and the code of the .aspx page.

In ASP.NET one retrieves the attribute value using expression Request.ServerVariables["SHIB_FOOBAR"]. It then can be used for information purposes or to show/hide parts of the web side, etc.

<% if (Request.ServerVariables["SHIB_CLEARANCE"] == "TopSecret") { %>
    <!-- show top secret stuff here -->
<% } %>

Note that this comes on top of any static protection that you might have: users whose attributes don’t match the rules from <RequestMap> in shibboleth2.xml will have their access denied and the dynamic application code won’t have a chance to run.

What’s next
I will discuss interesting issues I ran into when integrating my IdP with Shibboleth.

July 28, 2016

Git: how to switch to UNIX line endings on existing repository

I cloned a UINX-oriented repository, and git promptly converted CR to CRLF in all text files, which caused some problems down the line. Here’s how to revert to the original line endings:

1. Make sure there is no unsaved work – commit or stash anything that’s pending.

2. At the root of the repository create a file called .gitattributes and put the following text in it:
* -crlf
This ensures that no files are subject to the line endings conversion.

3. Clear your index. This will not physically remove any files, but will update git internal data structures (don’t forget that final dot):
git rm --cached -r .

4. Rebuild your working directory from the repository using new rules:
git reset --hard

Voila. Enjoy your untouched line endings.

July 25, 2016

Json.NET and dictionaries

TL;DR Json.NET can serialize and deserialize dictionaries with simple keys like integers or strings. It can serialize, but not deserialize dictionaries with more complex keys.

Longer version: Json.NET uses ToString() to convert a dictionary key to JSON field identifier. A simple case works like this:

[read more…]

July 23, 2016

Shibboleth Installation and Configuration

Previous: SAML and shibboleth
Next: Shibbleth: controlling access to resources

Shibboleth is an open-source SAML implementation that is used for single sign-on. We are developing a SAML IdP, and I was testing it against Shibboleth. Here are some pieces of information that I want to keep for my records. I used Shibboleth with IIS 8.


I downloaded the latest package from the the download page. At the time of writing it was shibboleth-sp- The installer targets IIS 6, so I had to perform manual steps outlined in the IIS 7 Installer page.

The scripts on that page more or less work, but watch out for that extra new line before “/+[path”: you will need to remove it for the script to work. If you don’t, you will get this error

Handler "Shibboleth" has a bad module "ManagedPipelineHandler" in its module list

If this happens, make sure to repeat the manual steps for mapping .sso extension to Shibboleth filter.

After the installation is done, Shibboleth will protect the content of the http://yourhost/secure virtual directory.


My Shibboleth configuration file. Note: this is not the actual file we use, some company specific information was removed.

Shibboleth configuration is stored in the shibboleth2.xml file. Standard configuration process involves going through the file and replacing default values such as host names with our specific data. However, default configuration uses SAML discovery protocol, which we do not support, so I had to perform more significant modifications:

  1. Copy IdP’s SAML metadata to Shibboleth configuration directory (C:\opt\shibboleth-sp\etc\shibboleth).
  2. Add the following node under <ApplicationDefaults>:
    <MetadataProvider type="XML" file="YourIdpMetadata.xml"/>
  3. Remove <SSO> and <Logout> nodes under ApplicationDefaults/Sessions.
  4. Add <SessionInitiator> node in their place:
    <SessionInitiator type="SAML2" Location="/Login" isDefault="true" id="Intranet"
    relayState="cookie" entityID="https://your.idp/EntityId">

    I am not certain what is the role of the “location” attribute. It appears to be ignored: SAML requests are sent to the URL in the metadata.
  5. If you leave it at that, Shibboleth will come up with a very cryptic message when you try to access http://yourhost/secure:
    Unable to locate a SAML 2.0 ACS endpoint to use for response.
    “ACS” here stands for “Assertion Consumer Service”. To get rid of the error, add the following node under <SessionInitiator>:

    <md:AssertionConsumerService Location="/SAML2/POST" index="1" 
    Binding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST" conf:ignoreNoPassive="true" />

    The documentation claims that an example is distributed with default shibboleth2.xml, but it is no longer the case. However, some examples can be found in the example-shibboleth2.xml file.

Restarting Shibboleth

Some configuration changes are picked up automatically, but for others you would have to restart Shibboleth. Run the following script as administrator

net stop shibd_default
net start shibd_default

Log level

Log levels are set in configuration file shibd.logger. This is a standard log4j configuration file, change “INFO” in the second line to “DEBUG” to get more detailed output. Don’t forget to restart Shibboleth after that.

What’s Next

I will discuss how to integrate Shibboleth SP with your IdP to control access to various resources.

July 22, 2016

Outlook glitch

A couple of hours ago my Outlook at work stopped working getting stuck on “trying to connect”. The Exchange Server appeared to be alive. Restarting and rebooting did not help.

I managed to unstick it by going into File -> Account Settings -> Account Settings (button) -> Double click on account -> More settings -> Security -> Always prompt for logon credentials (wow, what a way to structure the settings dialog!).

Next time the Outlook ran it asked me for user name and password, I entered them and voila – now it can connect. After that I went back to the same setting and unchecked the box.

The weird part is that it did not give me “invalid credentials” prompt, it would just hung.