Editing SharePoint Pages Using Visual Studio And WebDAV

I blog in a few different places.  I have my personal blog, my professional blog, and I maintain a blog at work to inform and educate co-workers.  At work, the blog is hosted on the company’s SharePoint server, which is fine.  I am still able to use Windows Live Writer and with it, the Insert Code plug-in.  My other two blogs use WordPress.

The Insert Code plug-in is invaluable to me because it does nice color coding of the text.  As part of that feature, it inserts a CSS style block into your post.  SharePoint doesn’t play well with this.  It tries to, but fails.  The intent is good.  SharePoint wraps your whole post in a div and gives it a class with a random name, then it rewrites the CSS styles so the classes will be scoped to only that containing div class.  Pretty smart way of encapsulating the styles.

Unfortunately, it fails on two points.  First, the containing div’s class is not like class=”123456789abcd”, it always precedes the class name with “ExternalClass”, so you get class=”ExternalClass123456789abcd”.  The rewritten CSS does not have any mention of “ExternalClass”.

The second mistake is in the rewritten CSS.  Your post will have a style block rewritten similar to:

<style>
123456789abcd h1 {color: red;}
123456789abcd h3 {color: green;}
123456789abcd .bold {font-weight: bold;}
</style>

Do you see the problem?  The class is 123456789abcd on the div (actually ExternalClass123456789abcd), but the stylesheet doesn’t scope it to that class.  Those are html tags it’s defining.  The stylesheet is looking for html tags of <123456789abcd>.

So, what can you do about this?  My solution was to put the stylesheet right in the template page so all the posts will be able to use those classes – that’s why I created this post.  The problem is, I couldn’t find anything in the SharePoint control panel to add a custom stylesheet (unlike WordPress, right?).  There was an option to edit the page using SharePoint Designer, so I installed Designer, only to find out the administrator didn’t allow editing using Designer. 

Other places on the Internet suggested adding a Content Editor web part and put the style sheet in there.  I tried it half-heartedly and gave up because it seemed way too “hack-y”.  But while doing so, it reminded me of something I used to know about SharePoint, that you could browse the site’s files using WebDav (assuming you had permissions).

So, what I did was map a network drive (a command found in many places in Windows Explorer) and gave it the URL of my SharePoint site.  Right away, I got an Explorer window with the template files.  I edited Default.aspx and Post.aspx and added my stylesheet.  The formatting was immediately applied.  Then I edited all my previous posts and removed my inline style code blocks to save space and reduce complexity.  Everything works now.

Visual Studio 2015 + Silverlight In VB.NET

At work, I was the first to upgrade my machine to Windows 10 and Visual Studio 2015.  Opening our application’s solution, I started resolving little errors here and there.  One problem was not so simple and had a very obscure solution.

We have a small collection of Silverlight apps written in VB.NET.  With the upgrade to Visual Studio 2015, they would not compile.  The errors returned were:

Microsoft (R) Visual Basic Compiler version 1.0.0.50618
Copyright (C) Microsoft Corporation. All rights reserved.

vbc : error BC30002: Type ‘Global.Microsoft.VisualBasic.ApplicationServices.ApplicationBase’ is not defined.
vbc : error BC30002: Type ‘Global.Microsoft.VisualBasic.Devices.Computer’ is not defined.
vbc : error BC30002: Type ‘Global.Microsoft.VisualBasic.MyServices.Internal.ContextValue’ is not defined.
vbc : error BC30002: Type ‘Global.Microsoft.VisualBasic.ApplicationServices.User’ is not defined.
vbc : error BC30002: Type ‘Global.System.ComponentModel.Design.HelpKeyword’ is not defined.

In the course of troubleshooting, I excluded everything from the project except the AssemblyInfo file and I still got the errors.  So I created a new Silverlight project, which built fine, and compared the Detailed compiler output.  The compiler line in the 2015-created project had a slightly different parameter (difference highlighted):

/define:"CONFIG=\"Debug\",DEBUG=-1,TRACE=-1,_MyType=\"Empty\",PLATFORM=\"AnyCPU\",SILVERLIGHT=1"

This clicked with me from my experience with building Silverlight apps for the Zune, where you had to exclude the My namespace that is created with VB.NET applications.  Look at the assemblies being referenced.  ApplicationServices.ApplicationBase (My.Application), Devices.Computer (My.Computer), ApplicationServices.User (My.User).

So, how do you get this extra compile setting?  It’s in the .vbproj – there’s no UI for it.  You’ll have to edit the vbproj file directly.

Unload the Silverlight project, right-click it, and choose Edit .vbproj.  Look for the node <NoStdLib>true</NoStdLib> (there will be one per project configuration) and after it, add: <MyType>Empty</MyType>.  This node is included with new Silverlight projects in VS2015.

Save and reload the project and the MyType flag will be included in the vbc command line, which will allow the Silverlight application to compile.

Of course, a lot of people will probably create a new Silverlight project and move all their code into it and be just as successful.  But, if you don’t want that hassle, or if you just want to know why (as I did), there’s the solution.

Windows 10 Groove Music – Zune On

Overview

With the release of Windows 10 comes a new music application, Groove Music.  Groove Music has Zune DNA, except that it loses any Windows Media Player (WMP) or Zune syncing capability.  The assumption is that the mobile phone is the new MP3 player and file copy is the preferred method of syncing.  For better or worse, this is the new normal.

Groove is much closer to the aesthetics of Zune than of WMP, and aside from the lost syncing capability and the toned-down Now Playing screen, it’s a reasonable Zune replacement – as a music player.  Syncing, well… not as much.  You have your usual views: Artist, Album, Song, Playlists, plus Albums for an Artist.  Genre view is missing.  Typing will expand the hamburger menu and put the text in the search box, proving immediate search.  Of course you have the Marketplace, to purchase and download more content.

Technical Details

Groove is a successor to Zune, although the outward branding does not hint at it.  The code library is called ZuneMusic and is found at %userprofile%\AppData\Local\Packages\Microsoft.ZuneMusic_8wekyb3d8bbwe.  In the subfolder LocalState you will find plenty of runtime details.  LocalState has a folder for the database, which is in ESENT format.  There are the ImageCache and imageStore folders that hold album artwork and artists photos from the Zune web services.

As far as the database is concerned, it seems to be similar if not exact to the old Zune database, which was in SQL Server Compact format.  The most common tables would be: tblAudioAlbum, tblPerson, tblGenre, and tblTrack to hold the music metadata and tblFolder and tblFile to hold the physical file references.

There are utilities and libraries to work with ESENT databases.  One is called ESENT Workbench.  If you do want to play around with the database, you may need to do a repair on it because it may not have shut down cleanly.  You can run the command “esentutl.exe /p EntClientDb.edb” to clean up the files for reading.

The Groove Music app uses another couple of packages extensively: Microsoft.Windows.CloudExperienceHost and Microsoft.Windows.ContentDeliveryManager, but probably not for primary functions.  The majority of data is likely pulled from the Zune web services, since the entries in the matadata database tables have references to GUIDs that, when used with the web services, retrieve the proper artist or album info.

The database for Groove Music appears to sync between computers, which makes a lot of sense for cloud-based music, but may get hairy when different machines have different local files.

Extension Ideas

What can be done with having access to the music database?  My impetus for this research was trying to change the Now playing slideshow to use all the artist pictures like Zune does, instead of a single album picture.  I haven’t gotten that far yet.  But some ideas for apps would be:

  • Statistic app showing most played artists, albums, songs, genres
  • Smart playlist generator based on statistics
  • Statistics on files: sizes, bitrates, dates, and something intriguing called FingerprintData
  • Utility to clean, purge, delete, export library
  • Post Now Playing, Recently Played to social media
  • Create a smart sync utility that utilizes the library’s metadata with file copy

Azure Aftermath

After the mess with the Azure SDK reserving ports that conflicted with my development environment, and the fact that the SDK didn’t do anything for the automated publishing I was looking for, I decided to uninstall it.  Well, that wasn’t a good idea.  You could say it was “Azure disaster.”

First off, the uninstall of the multiple elements of the Azure SDK did not remove the NETSH entries for the Azure Storage Emulator. so those had to be manually removed. Then, I reopened VS 2013 and suddenly none of my web projects open, saying they are all incompatible with this version of Visual Studio.  I tried to create a new web application and got the useful error:

image

So now I can’t open or create any web applications in Visual Studio.  I decide I should reinstall the Azure SDK to put back what was taken away, although in my mind I realize this is a ridiculous concept.

I download the SDK and run it.  I then get this error from the installer:

image

Absolutely incredible.  This is like dealing with anti-psychotic medication.  “Whoa whoa whoa, you just can’t uninstall Pericyazine, that can cause a psychotic episode.”

Next thing to do is a Repair of the Visual Studio installation, which is going to be at least an hour…  And that has fixed the web project issue, so at least uninstalling the Azure SDK is recoverable.  You just need to make sure you manually get rid of the ACL entries with these statements (at an Administrator command prompt):

netsh http delete urlacl url=http://127.0.0.1:10000/
netsh http delete urlacl url=http://127.0.0.1:10001/
netsh http delete urlacl url=http://127.0.0.1:10002/

Yes, I do believe when Windows 10 and VS 2015 come out, this work machine is due for a reformat.

IISExpress – Error description: Access is denied. (0x80070005)

Today, I spent my entire work day trying to find the solution to this error.

When I would launch a web project from Visual Studio, IIS Express would fail to load.  The error reported was:

Failed to register URL "http://localhost:10000/" for site "Portal3" application "/". Error description: Access is denied. (0x80070005)

I started down the obvious track of checking permissions.  IISExpress would operate just fine when run as Administrator, but not as my normal user.  I was an administrator on the local machine, so that didn’t make any sense.

The only thing I had done that morning was use Disk Cleanup to delete all my temp files and archived error reports.  I checked %temp%\iisexpress.  I checked my *.config files in Documents\IISExpress over and over.  I restored configs from backups.  I read many potential solutions, some as crazy as not being able to use a user name with “bg” in it.

I tried an older iteration of the web application.  It worked.  I tried other web projects in the solution.  They worked.  At this point, I was the only developer in my team who had a single web project that wouldn’t launch.

Then, late as usual, I finally remembered the Windows Event Viewer.  IISExpress was logging these two events:

Event ID: 2269: The worker process for app pool ‘Clr4IntegratedAppPool’, PID=’7536′, failed to initialize the http.sys communication when asked to start processing http requests and therefore will be considered ill by W3SVC and terminated.  The data field contains the error number.

EventID 2276: The worker process failed to initialize correctly and therefore could not be started.  The data is the error.

That lead me down a totally different troubleshooting path.  While doing so, I set up a new website in IIS on port 8085.  Oddly, my new websites and application pools didn’t refresh after being created, which led me to think there was even more wrong with my workstation.  An IISRESET resolved that problem and I was able to browse to the web project using IIS.

So IIS would work on port 8085 but IISExpress would not work on port 10000.  And IIS would work for any other project on seemingly any other port. 

In a comment on one of the web posts that I read, I saw this command:

netsh http add urlacl url=http://localhost:10000/ user=everyone

I’d only ever used NETSH maybe once or twice for some obscure reason, but the inclusion of “ACL” in the command was encouraging.  Amazingly, the command worked!  Although I was happy, I was also disappointed that I wouldn’t be able to see what the value was previously so I could find out the root cause.

So I ran:

netsh http show urlacl

And that displayed a bunch of entries that included these as well (User and SDDL changed):

Reserved URL            : http://127.0.0.1:10000/
    User: Me 
        Listen: Yes
        Delegate: No
        SDDL: D:(A;;GX;;;S-1-5-21-)

Reserved URL            : http://127.0.0.1:10001/
    User: Me
        Listen: Yes
        Delegate: No
        SDDL: D:(A;;GX;;;S-1-5-21-

Reserved URL            : http://127.0.0.1:10002/
    User: Me
        Listen: Yes
        Delegate: No
        SDDL: D:(A;;GX;;;S-1-5-21-)

Where did these come from?  I ran netsh http delete urlacl url=http://localhost:10000/ and removed the entry and confirmed IISExpress stopped working.  Then I ran netsh http delete urlacl url=http://127.0.0.1:10000/ and IISExpress started working again.

Something had added ACL entries for ports 10000-10002 that were conflicting with my web project, which was trying to run on port 10000 using IISExpress.  How did they get there?  I looked in Add/Remove Programs to get a clue as to what could have added these entries.  Who’s to blame?  The Azure SDK.  Azure Storage Emulator uses ports 10000-10002 and creates reservations for them.  This was installed during my attempts at getting command-line Web Publishing to work.

I never would have known anything about this unless I had read the Azure SDK documentation.  The error message I was given said nothing about port conflicts.  nothing led me down that path at all.  And it’s entirely possible no one else would have this problem unless they were running IISExpress on port 10000.

But the important takeaway from this is that NETSH will allow you to create reservations for port numbers that may conflict with other applications.

MSBuild error MSB4057: The target “Package” does not exist in the project.

Every three weeks we release an update to our websites and web services.  To make this release easier, I created a batch file that would build the projects and deploy each one to our four web servers.  The last few times I tried this, my batch file failed running this command:

c:\Windows\Microsoft.NET\Framework\v4.0.30319\MSBuild.exe C:\Projects\Portal\Portal1.vbproj /nologo /verbosity:minimal /t:Package /p:Configuration=Release

C:\Projects\Portal\Portal1.vbproj : error MSB4057: The target "Package" does not exist in the project.

So then each time, I would have to manually deploy the web sites with One-Click Publish.  Today, I decided to resolve this problem.

Because the Internet has a great memory and because the nature of deploying Visual Studio has changed frequently and recently, it was very difficult to determine what was the current best way to do the automated task that I wanted. 

The first promising solution was to install a project called “CommunityTasks” and import it into your project.  Did that.  Didn’t work.  Read further and learned I needed to install the Azure SDK (This would haunt me for a long time).  Still, none of the example command lines worked.

Then I learned that some publishing settings had been moved from the project file to the publishing profiles.  Fine, I could handle that.  I created a new publishing profile that created a package.  However, I couldn’t figure out how to execute that publishing profile from the command line.

In the end, I decided I would create the deployment packages manually in VS with One-Click Publish, then execute a batch file that would run the package’s deploy.cmd files for each project to each server.  This would actually result in a faster deployment because I wouldn’t have to wait for each project to build in the deployment batch file.  And using the /k switch, I could launch multiple deployments at once.  For example:

start cmd /k "Portal1.bat"
start cmd /k "Portal2.bat"
start cmd /k "Portal3.bat"
start cmd /k "Portal4.bat"

And each batch file for the project would install to each server:

c:\Build\Package\portal1.deploy.cmd /Y /M:http://server01/MSDeployAgentService /U:deploy /P:deploy -enableRule:DoNotDeleteRule
c:\Build\Package\portal1.deploy.cmd /Y /M:
http://server02/MSDeployAgentService /U:deploy /P:deploy -enableRule:DoNotDeleteRule
c:\Build\Package\portal1.deploy.cmd /Y /M:
http://server03/MSDeployAgentService /U:deploy /P:deploy -enableRule:DoNotDeleteRule
c:\Build\Package\portal1.deploy.cmd /Y /M:
http://server04/MSDeployAgentService /U:deploy /P:deploy -enableRule:DoNotDeleteRule

Still Not Giving In

I’m still using MS Money.  And I’ve come across a couple of instances of it beginning to lose compatibility with modern systems.  So now, I’ve actually started creating workarounds for them.

I’ve used a variety of online accounts in my many years.  I’ve used HSBC, Capital One (the non-360 variant), Sallie Mae, and most recently, Ally.  At this point, I’ve decided Ally is getting all my business and I’ve been in the process of moving accounts into new Ally subaccounts, which is very easily done on their part.  Just today, I discovered the transaction download feature.  There’s no MS Money OFX option, but I don’t think Money existed anymore when Ally came on the scene.  Anyway, there is a Quicken download, so that is what I use.

MS Money is awesome in that it supports QFX files, however, the standard format of the file must have moved on in time, so now Money throws up when it tries to process the file.  After a bunch of trial and error, I discovered that the reason for the error is a node in each transaction entry for the check number: <CHECKNUM>0</CHECKNUM>. Once you strip that node out, the file imports just fine. 

In another case, my 401k provider, Transamerica, recently revamped their transaction download and their QFX files have a different problem.  The file headers look like:

OFXHEADER: 100
DATA: OFXSGML
VERSION: 102
SECURITY: NONE
ENCODING: USASCII
CHARSET: 1252
COMPRESSION: NONE
OLDFILEUID: NONE
NEWFILEUID: NONE

But there is a space after the colon, which causes MS Money to report the file is corrupt.  The header should look like:

OFXHEADER:100
DATA:OFXSGML
VERSION:102
SECURITY:NONE
ENCODING:USASCII
CHARSET:1252
COMPRESSION:NONE
OLDFILEUID:NONE
NEWFILEUID:NONE

So I made a script that will alter the QFX file and then launch the Money importer.  All you have to do is drag the QFX file onto the VBS file and you’re good to go.  If you want to get clever, you can put the script in your SendTo folder or map it as a default application.

Without further adieu, this is the content of the script:

dim fso,f,s,shell

set fso=CreateObject("scripting.filesystemobject")

set f=fso.OpenTextFile(WScript.Arguments(0),1)
s=f.ReadAll
f.close
set f=nothing

set f=fso.OpenTextFile(WScript.Arguments(0),2)
f.Write Replace(Replace(s,"<CHECKNUM>0</CHECKNUM>",""),": ",":")
f.Close
set f=nothing

set fso=nothing

Set shell = CreateObject("Shell.Application")
shell.ShellExecute "C:\Program Files (x86)\Microsoft Money Plus\MNYCoreFiles\mnyimprt.exe",  WScript.Arguments(0)
set shell=nothing

And then, you can import QFX files from Ally or Transamerica (and maybe some others that have the same problems) into MS Money without any errors.

Two-Factor Authentication Primer

I recently implemented two-factor authentication into a web app and since it was a new concept for me, I thought it would be good to explain the highest conceptual level of this process.  As with a lot of new things, there’s some terminology to learn and there’s a need to understand how all the pieces fit together.

First, what’s it take to integrate this with an existing profile login?  You need a new database field and a bit of extra code for opting in and out of the two-factor authentication.  Ideally, you’ll need a library for generating a QR code, too.

Before I get too much into it, these are some of the elements of the process.  There are three pieces of data involved:

  • Shared Secret: This element is stored in the database with the user profile and is never exposed outside your application.
  • Secret Key: This is an encoded version of the Shared Secret.  It is given to the user by your application and the user enters it into their authenticator application.
  • Code: The numeric value generated by the authenticator application.  This changes every minute.

In brief, your application and the authenticator application both use the current time plus the Secret Key to generate a Code.  If they match, the user is authenticated.

To implement this, you would modify your user profile page to provide a button to enable two-factor.  When the button is clicked, you create a a random Shared Secret and save it to their profile.  You use that Shared Secret to generate and return the Secret Key.  The user puts that Secret Key in their authenticator app and the opt-in is complete.

When the user logs in to your application, if they have a Shared Secret set in their profile, they are prompted to enter the Code from their authenticator app.  Your application compares that Code to the Code it generates itself, using the Secret Key (built from the Shared Secret).  If it is the same, the user is logged in.

It really is simple.  The only thing that isn’t clear, but can be found with some moderate Internet searching is the URL to embed in the QR code.  That URL is: otpauth://totp/{0}?secret={1}, where {0} is the name of the profile to use (either your application or the user’s username or both) and {1} is the Secret Key.  Authenticator apps allow manual entry of Secret Keys, so if you don’t provide a QR code, it’s still workable, just a bit tedious.

Some of the other pieces you’d need are functions to reset the Shared Secret or clear it, if the user wanted to opt-out.  This is simple user account maintenance.  With a simple implementation, you could blank out the Shared Secret on a “forgot password” action.  With more sensitive data, you may want a second code to allow a password reset.  The big concern is users who have lost their phone or wiped out their authenticator application entries.

Because two-factor authentication is so simple and is such a low-impact to existing user profile data structures (relative to oAuth), plus the fact it can be opt-in, it’s really a no-brainer to add it to your applications.

PHP Hacked Site

While doing a search for something innocuous, I found a search result that was very out of place.  The domain was nothing related to what I was searching for, and the text abstract was, to say the least, spammy.  Although I know you’re not supposed to click things like that, I figure I’m pretty secure, so I clicked it.

I was immediately shown a page that said my download would start in 0 seconds, then I was prompted to download an EXE file.  Uh huh.  I browsed to the root domain and it really was a legitimate website.  So now, I wanted to figure out how this happened.  I navigated to the hacked page and I didn’t get any download prompt.  I went back to the search results and clicked again – I got the download prompt.  Hmmm.  More attempts and sometimes the site would send me to a dead page.

image

I looked very hard at the source code and couldn’t find the script that was being injected, but I could see there was a comment <!–counter–> that was getting replaced with the download redirect.  I did a site search on Bing and found many, many, many pages on their website that were suspect.  Also, I saw actual website pages that were in PHP.

So, I had to conclude that the website had a hacked version of PHP, and if that was compromised, the server could do anything it wanted, including checking for referrers and replacing tags in the source code files.  The best I could do was email them and let them know they were hacked and that they had to have their webmaster fix it for them.

Upon further research, it looks like it was a Joomla exploit from a couple of years ago.  I passed that info along and hopefully the website owners can make the updates needed (and clean up all the extra pages).

SSRS ReportViewer NullReferenceException on Dispose

I recently assisted on troubleshooting an error in a utility application where an exception was being thrown on the dispose of a Microsoft.Reporting.WebForms.ReportViewer.  The environmental conditions were pretty specific, so it’s possible you’d never see something like this in your environment.  But if you do, here’s how you can work around it.

The specific condition is that we have a shared library of code for both desktop and web applications.  One of the functions in that library takes some parameters for an SSRS report and returns a byte array for a rendered PDF of the report.  Because the library initially was used exclusively by the website, the WebForms version of the ReportViewer was used.  As time went on, the library was used by desktop apps and windows services.  That’s when the trouble began.

So, if you are using a WebForms.ReportViewer in a desktop application, you may get this exception when disposing the instance.  Digging into the decompiled code for the ReportViewer control suggested it was because there was no HttpContext available.  For us, the long-term fix was clear: use the WinForms version of the ReportViewer.  In the short term though, adding this line of code resolved the error:

If HttpContext.Current Is Nothing Then HttpContext.Current = New HttpContext(New HttpRequest(IO.Path.GetRandomFileName, "http://www.google.com", ""), New HttpResponse(IO.TextWriter.Null))

This created an HttpContext where there was none before, and the ReportViewer instance was able to be disposed without an error.