HOW TO: Implementing a Backup Strategy with OOB STSADM commands

 

STSADM Backup Syntax:

For site collection backup

stsadm -o backup

-url<URL name>

-filename<file name>

[-overwrite]

Things to look out for when using STSADM to perform backups

As your site grows, logically does your content databases, and if you are employing STSADM as the tool for backup, a bi-product of that will be “longer backup times”. If your environment will be taking longer to backup, your risk of someone updating a file/document/asset as you are performing your backup increases; as a result if a backup is in running and a file is open, that backup is at risk of failure. How do we combat that problem? We use yet another STSADM command to lock the Site in effect the database to read-only; we do this with the SiteLock command, there are two in particular, getsitelock and setsitelock see below for the full command syntax.

stsadm -o getsitelock

-url <URL of Site Collection>

stsadm -o setsitelock

-url < URL of Site Collection> -lock readonly

Automate your Backup Solution

Certainly you can navigate to the SharePoint 12-Hive and execute these commands as a single action; however, a more consistent action is to script the job with error handling and user feedback.

Script your Backup

@echo off
@echo——————————————————–
@echo This script will backup your site: [Your Site Name Here]
@echo——————————————————–
cd \Program Files\Common Files\Microsoft Shared\web server extensions\12\BIN
@echo locking site collection before the backup process starts
@echo off

stsadm -o getsitelock -url http://YourSiteCollectionNameHere
stsadm -o setsitelock -url http://YourSiteCollectionNameHere  -lock readonly
@echo site collection is now locked

@Echo Logic to create a new file name daily 
@For /F "tokens=1,2,3 delims=/ " %%A in (‘Date /t’) do @(
Set Day=%%A
Set Month=%%B
Set Year=%%C
Set All=%%C%%B%%A
)

@echo off

stsadm -o backup -url http://YourSiteCollectionNameHere  -filename \\ServerName\ShareName\Backup_ %All%.bak -overwrite
@echo backup completed
@echo off
stsadm -o setsitelock -url http://YourSiteCollectionNameHere  -lock none
@echo Site lock removed

Save the above to a .bat file in a preferred location.

Automate your Backup

Create a new Windows Scheduled Task by going to Start–> Control Panel –> Scheduled Tasks –> Add Scheduled Task

Follow the wizard and schedule the .bat file created above to run at a frequency of your own choosing.

How to Build a SharePoint Development Machine

 

There are two choices: You can build a SharePoint server that contains all your developer tools too. Or you can build your SharePoint server in a virtual machine (VM or VPC), and keep all your developer tools local. Then you can treat your virtual machine just like a physical server, and surf or demo it just as you would a production environment. This way you avoid all the inconsistencies that come with running browser session on locked-down web servers, don’t fill your server’s GAC with quite as much junk, and really get the client experience. You can use this post as a starting point to build either.

This post describes the tools you will use. There are always others, so go ahead and tell me about yours in the comments section. I’ve left off CSS and XML editors because frankly the best aren’t free. Even though a few items listed here are commercial products (e.g. Visual Studio), all have evaluation versions that last long enough for you to learn the ropes and prepare for that first big SharePoint interview.

Client Hardware

The biggest controllable factors on the speed of your SharePoint development environment are RAM and hard drive speed. If you do a lot of development, you will also want a lot of storage space to hold all those virtual machines.

The configuration described here is intended for a good laptop, which today is limited to 4Gb RAM. If you have 4Gb, give the virtual machine 2Gb. If you have 3Gb, give the virtual machine either 1.5Gb or 2Gb. If you have 2Gb or less, buy more RAM or host your virtual server elsewhere. Seriously.

If you need Exchange Server for any reason (1 to 2Gb recommended), you’ve grown out of laptop labs and should build this on a Virtual Server (or VMWare) host if you want some hair left over when you’re done. To build medium or large farms in a virtual environment, Virtual Server running on a 64-bit Windows 2003 Server R2 machine will remove your memory limitations. A 64-bit Vista machine provides the same expandability with Virtual PC or VMWare.

There are several factors to hard drive speed: keeping your VM on a different drive than your OS, drive transfer rates, and drive speed. Put another way: you want a big fast external drive. You can get a pretty big drive for around $200CAD now (I like my Seagate), and an enclosure with Firewire or eSATA (external SATA) is about $50CAD. If you don’t have Firewire or eSATA on your laptop you’ll need an adapter for that as well (also $50). Why not USB 2.0? Firewire is 33% to 70% faster than USB depending on the operation. eSATA is about five times faster than USB 2.0. While many enclosures also have a USB port, the difference is well worth it. PCMCIA (CardBus) runs at 132Mbytes per sec so an adapter for any of these will treat you fine. Note that newer laptops may have an ExpressCard slot instead of a PCMCIA slot.

Technology	Transfer Rate (Mbits per second)
USB 2.0	480
Firewire 800	786
eSATA	1500 to 2400

Minimal Configurations

The assumption is that you know what you like, know what you want to do, and will download and install accordingly. That said, maybe you really do want to develop with Notepad so here goes.

On your development machine you need the .Net Framework version 3.0 and a development interface (IDE), whether it’s Visual Studio, SharePoint Designer, or a plaintext editor like Notepad. If you don’t have a physical server available, you will use either Virtual PC or VMWare to host SharePoint. Beyond these, everything is optional.

On your server, you need Windows Server, IIS, the .Net Framework version 3.0, and either WSS, MOSS Standard, or MOSS Enterprise.

Building the Client

.Net Framework 2.0. This is optional, but if you do any development beyond SharePoint you likely have clients on v2.0.

.Net Framework 3.5
WPF, WCF, and essential to SharePoint: Workflow. Note that MOSS 2007 works great with .NET 3.5, is supported by Microsoft, and there is no reason not to deploy it for all your servers.

[Update 2008-11-04] Do not install .NET 3.5 SP1 if you require Records Center, and until the cause is identified there is a risk that other features are also affected. .NET 3.5 itself remains okay, this issue is only reported with SP1.

Microsoft Office SharePoint Server 2007 SDK [Read it online]
The MOSS SDK includes the Enterprise Content Management (ECM) Starter Kit, a BDC definition editor, an Excel UDF sample, great workflow samples, and lots more.

Windows SharePoint Services (WSS) 3.0 Software Development Kit (SDK) [Read it online]
The MOSS SDK contains this so you don't need both. If you only do WSS development, then you only need the WSS SDK.

Red Gate .NET Reflector (was Lutz Roeder's Reflector)

One of the best .NET tools ever, and the way you'll browse object models and cut the class-assembly strings you'll need to paste into your manifests and web.config. “Reflector is a class browser, explorer, analyzer and documentation viewer for .NET. Reflector allows you to view, navigate, search, decompile and analyze .NET assemblies in C#, Visual Basic and IL.”

SharePoint Manager 2007
The best SharePoint management tool there is. Browse the complete object model, deactivate features, and even update object properties. You do need to be careful with this one, but wow it's powerful.

Office System Professional Plus Edition
Grab your disc, MSDN, or get the evaluation version. The Professional Plus edition includes Word, Excel, Outlook and PowerPoint plus: Access, Publisher and InfoPath. There is also an Enterprise edition which also includes OneNote and Groove.

SharePoint Designer (SPD)
Grab your disc, MSDN, or get the evaluation version.

Visual Studio 2005 or 2008
Grab your disc, MSDN, or get the evaluation version. I prefer either VS Professional or VSTS Developer Edition.

Visual Studio 2005 Extensions for .NET Framework 3.0 (Workflow Foundation)

Not required with VS 2008 where these features are baked-in. Design Windows Workflow Foundation solutions for SharePoint and other .NET applications.

Visual Studio 2005 Tools for the 2007 Microsoft Office System (VSTO)
VSTO is used to build: Office System (Word, Excel, etc.) Add-ins and Ribbon elements, Outlook forms, and InfoPath templates.

Other Visual Studio Extensions. Optional extensions are available unrelated to SharePoint, though you should know they exist if you do other .NET development.

Virtual PC 2007

This is where you will build all those beautiful webs! VMWare is a popular alternative.

SQL Server Express Edition
You may want SQL Express if you develop more than SharePoint and need a local data store. If not, it is built into the SharePoint standalone install so you do not need to download or install it locally.

Optional

Fiddler HTTP Debugger
Optional and strongly recommended. Inspect and manipulate HTTP traffic between your browser and the server, great for debugging forms and web services, determining whether Kerberos tokens are being generated, and more.

Internet Explorer Developer Toolbar
The IE Developer Toolbar is optional, but recommended. Inspect HTML source, style usage, IFRAME content, image locations, and more.

PowerShell (formerly Monad). [SharePoint Provider] [Script Repository]
PowerShell is optional, but recommended. Write scripts to automate administration and configuration tasks. Colin Byrne has posts to get you started: PowerShell and SharePoint, Build Me A Portal, and Upload a Directory in 4 Lines.

BDC Metadata Manager
Generate BDC XML from SQL data sources. The commercial version also builds definitions for web services. An alternative is baked into the MOSS SDK, but BDC Metaman will continue to be the gold standard.

Visual Studio 2005 Extensions for Windows SharePoint Services 3.0 (VSeWSS)
Not recommended. VSeWSS contains nothing to help you deploy features on anything but the machine running VS, does not support x64, and is required to open projects that it creates. So if you create a project today on a 32-bit machine, you will be unable to open it in a year when you're building on 64-bit. This is bad. The one good feature included is the SharePoint Solution Generator (SPSolGen) which converts lists and sites “Saved as Template” into list and site definitions. [Learn more here] [Registry hack to install VSeWSS to a machine that is not the server]

Required to create a private LAN between the client and server

Microsoft Loopback Adapter

a. Click Start, Control Panel, and open the Add Hardware wizard.

b. Click Next. Wait a moment while the wizard scans for hardware.

c. Select: Yes, I have already connected the hardware, click Next.

d. Scroll to the bottom of the list and select Add a new hardware device. Click Next.

e. Select: Install the hardware that I manually select from a list (Advanced). Click Next.

f. Select: Network adapters. Select: Microsoft. Select: Microsoft Loopback Adapter.

g. Click Next. Click Next again. Wait a moment. Click Finish.

Configure the Loopback Adapter

h. Click Start, Control Panel, and open the Network Connections utility.

i. Right-click the Microsoft Loopback Adapter connection and select Properties.

j. Double-click Internet Protocol (TCP/IP).

k. Now configure an address for this adapter.
Select: Use the following address
IP Address: 10.50.50.1
Subnet mask: 255.255.255.0
Default gateway:
You can ignore DNS. Click OK. Click OK again.

Building the Server: Articles

Bob Fox explains step-by-step how to install Windows 2003 to Virtual PC and VMWare (up to the point where you install the OS).

Bob Fox explains good habits on re-using your base image for several implementations, and why you might need to add servers to the farm (e.g. for Exchange).

How to Create a MOSS 2007 VPC Image. This step-by-step series with screenshots by Tony Zink is as complete as complete can be and starts with configuration of IIS. Note that he puts SPD and Office on the server, while the goal of this post is to keep your server clean of client apps.

Microsoft Guidance on Standalone and Farm configurations.

Kevin Hoffman explains how to build a Standalone SharePoint server. He installs Visual Studio to the server, you can skip that step and run it locally on your development machine.

Jonathan Bradshaw explains how to build a Small Farm configuration with SQL and AD on a separate machine. His “OfficeDev” machine looks the most like our local development machine. One day someone will write an article with configuration steps, until then this is a good start.

Building the Server: Step-by-Step

Step 1: Install the Operating System

Windows 2003 Server R2. Grab your disc, MSDN, or get the evaluation version. Bring the machine up-to-date with Windows Update. Note that  this process can last several hours.

Step 2: Configure Networking

R. Aaron Zupancic describes how to configure an IP address and host file entry so you can surf your virtual SharePoint sites from your development machine.

Simon Guest describes how to get all the machines in a virtual farm to talk to each other. In this configuration, while your virtual machines will see each other, your local machine will not be able to surf or debug the virtual machines.

Note that if you can join your virtual machine to your company’s domain and create domain accounts there (but different accounts than you will use for testing or production), it will simplify configuration and debugging, and you won’t need to make this or any machine in your development farm an Active Directory domain controller (DC). The drawback is that you need to be connected to the domain to get any work done.

Step 3: Install the .NET Framework

.Net Framework 3.0
WPF, WCF, and essential to SharePoint: Workflow.

Once activated and configured, Shut Down and back up your base server image. In the next steps you’ll prepare this machine for a specific purpose.

Step 4: Install Active Directory and DNS.

If this is going to be a Standalone configuration, you need a domain controller (DC). The Configure Your Server Wizard (Start, Administrative Tools) is a great way to assign roles to a server. Start with Domain Controller (Active Directory) and DNS.

If this is going to be a Small Farm, copy your base server image from the last step, and start building a second machine to host Active Directory and SQL Server. Note that after you install SharePoint you will not be able to make the machine a domain controller (unless you’re prepared for a world of hurt).

Step 5: Install Internet Information Server (IIS)

Once again, the Configure Your Server Wizard (Start, Administrative Tools) is an easy way to add the Application Server role to your VM.

Once IIS is installed, you will need to Allow ASP.NET 2.0.* in the Web Server Extensions section of the IIS Manager. If this option is not available, you may need to run the ASP.NET 2.0 setup and Repair your installation, which also sets the Allow flag.

Step 6: Install SQL Server

SQL Server Developer Edition [Google]
This is optional. If you don’t install it, Express Edition will be installed when you install SharePoint. However, Developer Edition includes the excellent SQL Server Management Studio, so if you have MSDN you may prefer it. For a production environment you will need to license SQL Server Standard or Enterprise Edition.

Step 7: Install SharePoint (WSS or MOSS)

Windows SharePoint Services (WSS) 3.0
WSS is a free extension to Windows Server. If your installation didn’t include WSS as an option, you can download now. If installing MOSS, then you do not need to separately install WSS. Also remember that you can upgrade a server, but you cannot downgrade from MOSS back to WSS. If you’re evaluating SharePoint you may want to start with WSS.

If you installed SQL Server Developer Edition, choose Advanced Installation to select your database.

Install the WSS Service Pack.

Microsoft Office SharePoint Server (MOSS) 2007
Grab your disc or MSDN, or get the evaluation version. Standard and Enterprise editions exist, so again keep in mind that you can upgrade but not downgrade. Enterprise includes Forms Server and the BDC. 

Install the Office SharePoint Server Service Pack. [SP1]

Shut down and back up your virtual machine image(s). 

Step 8: Configure SharePoint

Create a domain group for SharePoint Administrators, assign rights to it rather than any single account. The exception will be configuring SQL Server, where rights buried inside groups aren't always resolved and you need to assign permissions per-account.  

Create an account to use as your SharePoint Service account (e.g. SPService). Configure this to be your Application Pool identity (you'll need to search what rights are required until I complete this section).

Create a content crawler account (you'll need to search what rights are required until I complete this section).

Create a developer account with the same name as the account you use on your development machine (and it better not be Administrator). Add your developer account to the Administrators group (Global Administrators if this is a DC, local Administrators if not) and Debugger Users group.

Create a typical end-user account for testing. Make it a member of Domain Users.

Step 9: Configure the server for Development

If you must develop on the server, install Visual Studio now. On one side, developing locally is simpler than developing remotely. On the other side, it’s so handy to keep all your code in one place and you won’t ever wonder where the most recent version of x is residing. If you really can’t get the hang of remote debugging, you can always install VS later.

Open up File Explorer on the server and Add to Favorites your frequently-used paths including the 12 hive and the default web root.
C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\
C:\inetpub\wwwroot\

Backup the 12 hive into a zip file marked as read-only, and named “[Drive]:\Backups\12Hive-Out-of-Box-DoNotUpdate.zip”. You get the idea. When you accidentally edit a default ONET.XML file with WordPad and kill your site you will be glad you did.

And finally, make a text file on your desktop with Joubin’s SharePoint paths.

Step 10: Configure remote debugging.

Once SharePoint is installed, share the folder with SharePoint’s assemblies (*.DLL). You may also want to copy the assemblies to your development machine:
C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\ISAPI

Share the server’s web root and enable Read/Write.
C:\inetpub\wwwroot\

Install the Visual Studio 2005 Debugging Components to the server. You want Full Remote Debugging, not Native Remote Debugging.

Follow Ross Dargan’s instructions on configuring remote debugging. You’ve just completed his step 1 and if you created your developer account as specified then you’ve also done 2 and 3.

Take a look at Andrew Noon’s document on Remote Debugging SharePoint Web Parts. He has additional steps for enabling error messages in web.config and screen shots that show what debugging looks like in action.

Read: Setup Remote Debugging

Read: Debugging SharePoint Web Parts

Downloading Content From SharePoint (Let me count the ways)

In this post I will be doing comparisons for downloading content from SharePoint. Once again there are many ways to get content from SharePoint. When I say content I am focusing on files and metadata. There is an additional way to get a list item using the Lists web service and the GetListItems method; however, this only returns metadata.

 

Method	Complexity	Scalability	Metadata	Versions
Copy.asmx	2	4	yes	no
WebDav	5	5	yes*	yes**
RPC	10	10	yes	yes

*must be used in conjunction with Lists web service

**must be used in conjunction with Versions web service

Copy Web Service

The copy web service is the easiest to use to get content from SharePoint because it is familiar web service programming and it allows the content and the metadata to be retrieved in one method call. The scalability is not as good as the other methods because it uses the more verbose soap protocol. Finally, one of the biggest disadvantages using the copy web service is the fact that it does not support returning versions of a document. The copy web service was made for migrating content between document libraries and sites within the same site collection not necessarily versions. I did try using the copy web service to retrieve an older version of a document by setting the url argument to the GetItem method using the older version’s url.

The version url looks like this:

http://servername/site/_vti_history/512/documentlibrary/filename.tiff

Unfortunately, when sending in a version url the GetItem method executes successfully but both the metadata and binary are empty.

public static void DownloadDocument()
{

copyservice.Copy c = new copyservice.Copy();
c.Url = "http://basesmcdev2/sites/tester1/_vti_bin/copy.asmx%22;
c.UseDefaultCredentials = true;

byte[] myBinary = new byte[] { };
copyservice.FieldInformation information = new copyservice.FieldInformation();
copyservice.FieldInformation[] info = { information };

string docUrl = "http://basesmcdev2/sites/tester1/Shared%20Documents/+aaa/mama.bmp%22;

uint result = c.GetItem(docUrl, out info, out myBinary);
if(File.Exists("c:\\newfile.bmp")) File.Delete("c:\\newfile.bmp");
using (Stream s = File.Create("c:\\newfile.bmp"))
{
s.Write(myBinary, 0, myBinary.Length);
s.Flush();
}
}

WebDav

Using webdav to download a document is fairly simple if you have the full url of the document. However, if you want to include metadata you must call the GetListItem of the list web service. This becomes tricky when the only piece of information you have is the url. You can use the list web service but you will need to know the name of the list and the id of the item in order to retrieve the metadata. In the end, if you are going to call a SharePoint web service then just call the copy web service and get both the metadata and file in one call.

public static void GetFileWithMetaData(string url)
{

WebClient wc = new WebClient();
wc.UseDefaultCredentials = true;

byte[] response = wc.DownloadData(url);

string returnStr = Encoding.UTF8.GetString(response);

if (File.Exists("c:\\default.txt")) File.Delete("c:\\default.txt");

using (Stream s = File.Create("c:\\detault.txt"))
{
s.Write(response, 0, response.Length);
s.Flush();
}

}

So how does WebDav support versions? You can give the DownLoadData method the url of the version mentioned previously and it will return the binary for that version. So, now the question is how to you obtain the url’s of previous versions? Below is a code snippet that uses the Versions web service to retrieve a particular version’s url for a document for a given file url and version number.

public static string GetVersionUrl(string url, double version)
{
double versionNumeric = 0;
string versionNumber = string.Empty;
string versionUrl = string.Empty;

versionservice.Versions vs = new versionservice.Versions();
vs.Url = "http://basesmcdev2/sites/tester1/_vti_bin/versions.asmx%22;
vs.UseDefaultCredentials = true;

XmlNode versionsNode = vs.GetVersions(url);

if (versionsNode != null)
{
using (StringReader sr = new StringReader(versionsNode.OuterXml))
{
using (XmlTextReader xtr = new XmlTextReader(sr))
{
XElement versionInfo = XElement.Load(xtr);

var versionResults = from r in versionInfo.Elements() where r.Name.LocalName == "result" select r;

foreach (XElement versionElement in versionResults)
{

var versionAttr = (from a in versionElement.Attributes() where a.Name == "version" select a);
versionNumber = versionAttr.First().Value;
versionUrl = versionElement.Attributes("url").First().Value.ToString();

//current version is in the form of "@1.0" and will not pass TryParse
if (versionAttr.Count() > 0 && double.TryParse(versionNumber, out versionNumeric))
if (version == versionNumeric) return versionUrl;
else
if (version == Convert.ToDouble(versionNumber.Substring(1))) return versionUrl;

}

}

}
}

return string.Empty;
}

FrontPage RPC (Remote Procedure Calls)

Most developers are not familiar with frontpage remote procedure calls. However, they are the most efficient and yet the most complex to code against. You could create your own wrapper classes to simplify coding. You must understand the command structure and be able to parse the return html correctly. In the case of downloading a document it is especially complex. You must come up with a way to parse out the returned metadata within the html. Below is an example of a successful return of a “get document” rpc call. Anything below the closing </html> tag is the file.

<html><head><title>vermeer RPC packet</title></head>
<body>
<p>method=get document:12.0.0.4518
<p>message=successfully retrieved document 'tester4/Code128.tif' from 'tester4/Code128.tif'
<p>document=
<ul>
<li>document_name=tester4/Code128.tif
<li>meta_info=
<ul>
<li>vti_rtag
<li>SWrt:FED298A3-6030-40DA-A984-D0A04A673741@00000000013
<li>vti_etag
<li>SW&#34;&#123;FED298A3-6030-40DA-A984-D0A04A673741&#125;,13&#34;
<li>vti_parserversion
<li>SR12.0.0.6318
<li>vti_modifiedby
<li>SRBASESMCDEV2&#92;steve.curran
<li>vti_filesize
<li>IR3387
<li>vti_timecreated
<li>TR22 Oct 2008 19:45:25 -0000
<li>ContentType
<li>SWDocument
<li>ContentTypeId
<li>SW0x01010087C42E0D80709D4CB61D6558C94571E4
<li>vti_title
<li>SW
<li>statechoice
<li>SWstateone
<li>vti_lastheight
<li>IX2200
<li>vti_timelastmodified
<li>TR13 Apr 2009 22:04:31 -0000
<li>vti_nexttolasttimemodified
<li>TR13 Apr 2009 22:02:20 -0000
<li>vti_candeleteversion
<li>BRtrue
<li>vti_canmaybeedit
<li>BXtrue
<li>vti_backlinkinfo
<li>VXLists/threeStateTasks/3_.000 Lists/threeStateTasks/4_.000
<li>myreq
<li>SWnbnbvnbv
<li>vti_author
<li>SRBASESMCDEV2&#92;steve.curran
<li>vti_lastwidth
<li>IX1696
<li>vti_sourcecontrolversion
<li>SRV1.0
<li>vti_sourcecontrolcookie
<li>SRfp_internal
<li>vti_level
<li>IR1
</ul>
</ul>
</body>
</html>

Finally, below is an example on how to call the “get document” rpc method. You must work with the returned byte array and copy the section which represents the file to another byte array. This method supports retrieving older versions. Just pass in the version number, zero would represent the current version. The fileUrl argument represents document library name plus the file name. For example, “Shared Documents/FileName.tiff”.

public static void DownloadDocumentRPC(string fileUrl, int version)
{

    string method = "get document: 12.0.0.4518";
    string serviceName = "http://basesmcdev2/sites/tester1/_vti_bin/_vti_aut/author.dll";
    string verstr = version > 0 ? "V" + version.ToString() : string.Empty;
    string document = fileUrl;
    byte[] data;
    string returnStr = string.Empty;
    byte[] fileBytes = null;

    string fpRPCCallStr = "method={0}&service_name={1}&document_name={2}&doc_version={3}&get_option={4}&timeout=0";

    method = HttpUtility.UrlEncode(method);
    fpRPCCallStr = String.Format(fpRPCCallStr, method, serviceName, document, verstr, "none");

    try
    {
        //add line feed character to delimit end of command
        byte[] fpRPCCall = System.Text.Encoding.UTF8.GetBytes(fpRPCCallStr + "\n");

        data = new byte[fpRPCCall.Length];
        fpRPCCall.CopyTo(data, 0);

        HttpWebRequest wReq = WebRequest.Create(serviceName) as HttpWebRequest;

        wReq.Credentials = System.Net.CredentialCache.DefaultCredentials;
        wReq.Method = "POST";
        wReq.ContentType = "application/x-vermeer-urlencoded";
        wReq.Headers.Add("X-Vermeer-Content-Type", "application/x-vermeer-urlencoded");
        wReq.ContentLength = fpRPCCall.Length;

        using (Stream requestStream = wReq.GetRequestStream())
        {

            requestStream.Write(fpRPCCall, 0, fpRPCCall.Length);

            int chunkSize = 2097152;
      

            //Now get the response from the server
            WebResponse response = wReq.GetResponse();
            int lastBytesRead, totalBytesRead;
            long contentLength = response.ContentLength;
            bool noLength = false;

            if (contentLength == -1)
            {
                noLength = true;
                contentLength = chunkSize;
            }

            byte[] returnBuffer = new byte[(int)contentLength];
            using (Stream responseStream = response.GetResponseStream())
            {
                totalBytesRead = 0;

                do
                {
                    lastBytesRead =
                        responseStream.Read(returnBuffer, totalBytesRead, ((int)contentLength) - totalBytesRead);
                    totalBytesRead += lastBytesRead;
                    if (noLength && (totalBytesRead == contentLength))
                    {
                        contentLength += chunkSize;
                        byte[] buffer2 = new byte[(int)contentLength];
                        Buffer.BlockCopy(returnBuffer, 0, buffer2, 0, totalBytesRead);
                        returnBuffer = buffer2;
                    }
                }
                while (lastBytesRead != 0);

            }

            if (noLength)
            {
                byte[] buffer3 = new byte[totalBytesRead];
                Buffer.BlockCopy(returnBuffer, 0, buffer3, 0, totalBytesRead);
                returnBuffer = buffer3;
            }

            returnStr = Encoding.UTF8.GetString(returnBuffer);

            //get begining of file bytes
            int startpos = returnStr.IndexOf("</html>") + 8;
            using (MemoryStream ms =
                new MemoryStream(returnBuffer, startpos, returnBuffer.Length - startpos))
                fileBytes = ms.ToArray();


            if (File.Exists("c:\\newfile.bmp")) File.Delete("c:\\newfile.bmp");
            using (Stream s = File.Create("c:\\newfile.bmp"))
            {
                s.Write(fileBytes, 0, fileBytes.Length);
                s.Flush();

            }



        }

    }
    catch (Exception ex)
    {
        //error handling

    }

}

Once again I hope this comparison of the different methods of downloading content from SharePoint will help you plan your next application’s SharePoint integration. Knowing the different methods will prevent you from having write your own web service. Ultimately, taking advantage of SharePoint “out of the box” tools will make your applications easier to install, configure and maintain.

Creating a custom webservice

Creating a Custom Web Service

1. The first step is to create an ASP.NET web service project in Visual Studio 2005. If you don't find a web service project template in visual studio, that means that you are still running an old version of Visual Studio 2005, the one without the service pack. You will have to download and install the Visual Studio 2005 Service Pack 1 Beta from the Microsoft. Here is the link:

http://www.microsoft.com/downloads/details.aspx?familyid=8D702463-674B-4978-9E22-C989130F6553&displaylang=en

It's a 371.9 MB download and let me tell you in advance that the installation is very slow and takes a lot of time.

On the File menu, click New Project.

2. In the Project Types box, select Visual C#.

3. In the Templates box, select ASP.NET Web Service Application.

4. In the Name box, type UploadService. In the Location box, type the following path:

C:\WebService

You can also click the browse button to browse the folders and select the destination folder. This is the path where the project will be stored.

In the Solution Name box, type UploadService and check Create directory for solution checkbox.

5. Click OK.

6. In the Solution Explorer, right-click Service1.asmx and rename the file Files.asmx and then right click Files.asmx and click View Code.

7. Add a reference to the assembly for Microsoft Office SharePoint Server 2007 (Microsoft.SharePoint.dll). This assembly is located in the following directory:

C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\ISAPI

Please note that if you are developing on a machine that does not have SharePoint installed then you can copy the required files from the SharePoint machine to your development machine.

8. Make sure following using directives are included at the top:

using Microsoft.SharePoint;
using Microsoft.SharePoint.WebControls;
using System.Net;

9. Change your class name from Service1 to Files.

public class Files : System.Web.Services.WebService

10. Comment out the web method definition. The commented out web method code will look like as follows:

//[WebMethod]
//public string HelloWorld()
//{
//return "Hello World";
//}

11. Add following code (web method) in the class:

[WebMethod]
public string UploadDocument(string fileName, byte[] fileContents, string pathFolder)
{
   if (fileContents == null)
   {
      return "Null Attachment";
   }
   try
   {
      int iStartIndex = pathFolder.LastIndexOf("/");
      string sitePath = pathFolder.Remove(iStartIndex);
      string folderName = pathFolder.Substring(iStartIndex + 1);
      SPSite site = new SPSite(sitePath);
      SPWeb web = site.OpenWeb();
      SPFolder folder = web.GetFolder(folderName);
      string fileURL = fileName;
      folder.Files.Add(fileURL, fileContents);
      if (folder.Files[fileURL].CheckedOutBy.Name != "")
      {
         folder.Files[fileURL].CheckIn("File Checked In");
      }
      return "File added successfully!";
   }
   catch (System.Exception ex)
   {
      return "Error: " + ex.Source + " - " + ex.Message;
   }
}

12. Open Files.asmx markup page. In Solution Explorer, right-click Files.asmx and select View Markup. You will notice that the markup page has following line:

<%@ WebService Language="C#" CodeBehind="Service1.asmx.cs" Class="UploadService.Service1" %>

Change it to the following line:

<%@ WebService Language="C#" Class="UploadService.Files" %>

13. Create a strong name for the class library. In Solution Explorer, right-click the web service project, and in the Properties dialog box, click Signing, select Sign the assembly, and select <New> in the box for choosing a strong name key file.

14. In the Create Strong Name Key dialog box, provide a file name for the key, deselect Protect my key file with a password, and click OK.

This was the easy way. You can also strong name your assembly using the command line utility called as sn.exe. Use following steps if you want to strong name your assembly manually:

1. Strong naming utility (sn.exe) can be found in the following folder:

C:\Program Files\Microsoft Visual Studio 8\SDK\v2.0\Bin

Copy the sn.exe utility to the folder where your assembly is located and run the following command to strong name your assembly:

sn.exe -k key.snk

Resulting key will be written to the key.snk file.

2. Go to your project's properties (right-click project name in the Solution Explorer and select Properties from the menu) and select Signing from the menu that appears on the left. This will open a form.

3. Check Sign the assembly checkbox and choose the key file from the drop down (Click the Browse... button in the drop down to locate the key.snk file that you generated in the previous step).

4. Re-compile your assembly.

15. Compile the web service project.

16. As we are developing this web service on a machine that does not host SharePoint, therefore, we need to create a virtual directory in IIS. Click Start, point to Administrative Tools (You may have to go to the Control Panel first to select Administrative Tools), and then click Internet Information Services (IIS) Manager.

Note: In case you are developing the service on a SharePoint server, then skip steps 17-20 and use following instructions and then start with step 22:

1. Copy the Files.asmx to the following folder:
C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\TEMPLATE\LAYOUTS

2. Run disco.exe at the command prompt from the LAYOUTS directory to generate .disco and .wsdl files.

Run a command in the following format to generate the files in LAYOUTS:

disco http://localhost/_layouts/Files.asmx

17.  Expand the branch for the server computer to which you want to add a virtual directory. Under the server computer branch, expand the Web Sites folder, and right-click the Default Web Site and select New and then Virtual Directory.... If you don't want to use Default Web Site, you can create a new web site.

18. Click Next and enter an alias in the text box, for example, for this service you can enter UploadService.

19. Click Next and then click Browse... button to browse to the project folder (containing the .asmx file) and click Next.

20. Click Next again and then click Finish.

Generating and Modifying Static Discovery and WSDL Files

21. Use Disco.exe to generate .disco and .wsdl files. This command line utility is located in the following directory:

C:\Program Files\Microsoft Visual Studio 8\SDK\v2.0\Bin

Open command prompt and type the following line and press Enter:

Disco http://localhost/uploadservice/Files.asmx

Make sure you have entered the correct path in the command above otherwise you will get an error. If you created a virtual directory on a port other than the port 80, then you must mention the port number in the path (For example, http://localhost:8080/uploadservice/Files.asmx). This will generate the .disco and .wsdl files.

22. To register namespaces of the Windows SharePoint Services object model, open both the .disco and .wsdl files and replace the opening XML processing instruction -- <?xml version="1.0" encoding="utf-8"?> -- with instructions such as the following:

<%@ Page Language="C#" Inherits="System.Web.UI.Page"%>
<%@ Assembly Name="Microsoft.SharePoint, Version=12.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c" %> 
<%@ Import Namespace="Microsoft.SharePoint.Utilities" %> 
<%@ Import Namespace="Microsoft.SharePoint" %>
<% Response.ContentType = "text/xml"; %>

23. In the .disco file, modify the contract reference and SOAP address tags to be like the following example, which replaces literal paths with code generated paths through use of the Microsoft.SharePoint.Utilities.SPEncode class, and which replaces the method name that is specified in the binding attribute:

<contractRef ref=<% SPHttpUtility.AddQuote(SPHttpUtility.HtmlEncode(SPWeb.OriginalBaseUrl(Request) + "?wsdl"),Response.Output=""); %>
docRef=<% SPHttpUtility.AddQuote(SPHttpUtility.HtmlEncode(SPWeb.OriginalBaseUrl(Request)),Response.Output); %>
xmlns="http://schemas.xmlsoap.org/disco/scl/" />
<soap address=<% SPHttpUtility.AddQuote(SPHttpUtility.HtmlEncode(SPWeb.OriginalBaseUrl(Request)),Response.Output); %>
xmlns:q1="http://tempuri.org" binding="q1:FilesSoap" xmlns="http://schemas.xmlsoap.org/disco/soap/" />
<soap address=<% SPHttpUtility.AddQuote(SPHttpUtility.HtmlEncode(SPWeb.OriginalBaseUrl(Request)),Response.Output); %>
xmlns:q2="http://tempuri.org" binding="q2:FilesSoap12" xmlns="http://schemas.xmlsoap.org/disco/soap/" />

24. In the .wsdl file, make the following, similar substitution for the SOAP address that is specified:

<soap:address location=<% SPHttpUtility.AddQuote(SPHttpUtility.HtmlEncode(SPWeb.OriginalBaseUrl(Request)),Response.Output); %> />

Make the following substitution for the SOAP12 address:

<soap12:address location=<% SPHttpUtility.AddQuote(SPHttpUtility.HtmlEncode(SPWeb.OriginalBaseUrl(Request)),Response.Output); %> />

25. Rename both files in the respective formats Filedisco.aspx and Fileswsdl.aspx so that your service is discoverable through SharePoint.

Deploying web service on the SharePoint server

Copy the Web service files to the _vti_bin virtual directory

26. Copy the web service files to the _vti_bin directory of the SharePoint server. Web service files that are to be copied are as following:

Files.asmx
Filesdisco.aspx
Fileswsdl.aspx

The _vti_bin virtual directory maps physically to the Local_Drive:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\12\ISAPI directory, which contains the default Web service files used in Windows SharePoint Services.

To include the Web service in the list of web services on the server

27. In Notepad, open the spsdisco.aspx file. spsdisco.aspx is located in the following directory (on the SharePoint server):

Local_Drive:\Program Files\Common Files\Microsoft Shared\web server extensions\12\ISAPI

28. Add the following lines to the end of the file within the discovery element and save the file:

<contractRef ref=<% SPHttpUtility.AddQuote(SPHttpUtility.HtmlEncode(spWeb.Url + "/_vti_bin/Files.asmx?wsdl"), Response.Output); %> docRef=<% SPHttpUtility.AddQuote(SPHttpUtility.HtmlEncode(spWeb.Url + "/_vti_bin/Files.asmx"), Response.Output); %> xmlns="http://schemas.xmlsoap.org/disco/scl/" />
<soap address=<% SPHttpUtility.AddQuote(SPHttpUtility.HtmlEncode(spWeb.Url + "/_vti_bin/Files.asmx"), Response.Output); %> xmlns:q1="http://schemas.microsoft.com/sharepoint/soap/directory/" binding="q1:FilesSoap" xmlns="http://schemas.xmlsoap.org/disco/soap/" />

Copying the assembly to the correct Bin folder

29. I have seen blogs and forums where people have recommended to copy the assembly to the following folder:

Local_Drive:\program files\common files\microsoft shared\web server extensions\12\isapi\

Some people say it should be copied to the following folder:

Local_Drive:\Inetpub\wwwroot\bin

bin folder is not there by default and has to be created but safest place to copy the assembly is the bin folder of the virtual directory of the web application where you intend to use the web service. Following path contains the virtual directories of web applications that you have created on your SharePoint server.

Local_Drive:\Inetpub\wwwroot\wss\VirtualDirectories

For example, if you want to use the web service in web application at port 81, then you should copy the assembly to the following folder:

Local_Drive:\Inetpub\wwwroot\wss\VirtualDirectories\81\bin

Similarly, if you want to use the web service in web application at port 17316, then you should copy the assembly to the following folder:

Local_Drive:\Inetpub\wwwroot\wss\VirtualDirectories\17316\bin

I am not saying that you can not copy the assembly to the _vti_bin folder or wwwroot folder, of course you can but I have often seen people struggling with the service deployment. People often ask me where they should copy the assembly and what is the best place to put the assembly in. I tested my web service by putting the assembly file in all the recommended places and I found that bin folder of the virtual directory of the web application is the safest place where your web service is guaranteed to work.

Copy the assembly to the bin folder of the virtual directory of the web application of your choice. You will have to create the bin folder yourself. It is not there by default.

Adding web service to the GAC

30. You must add your assembly in the GAC on your SharePoint server. This is necessary otherwise you will get permissions error on the server. There are three ways to avoid this permissions error. One way is to add the assembly in the GAC. Second way is to change the trust level to medium in the web.config file and the third is to create a custom trust policy of your own. There is an article on Microsoft site that has all the details of Code Access Security (CAS) and it also shows how to create a custom policy file. Here is the link to that article if you are interested in exploring this topic:

http://msdn2.microsoft.com/en-us/library/ms916855.aspx

Unfortunately, this is an old version and works only with SharePoint 2003. I am not sure if Microsoft has already released an updated version of this article or not. I intend to write an updated version of this article (at least the custom policy file part) for Office SharePoint Server 2007 and Windows SharePoint Services 3.0.

To add your assembly to the Global Assembly Cache (GAC), you can either drag and drop the assembly into the %windows%\assembly directory using 2 instances of Windows Explorer, or use the command line utility gacutil.exe that is installed with the .NET Framework SDK 2.0. This utility is located in the following folder:

C:\Program Files\Microsoft Visual Studio 8\SDK\v2.0\Bin

To use gacutil.exe to copy the class library DLL into the GAC

1. To open the Visual Studio command prompt, click Start, point to All Programs, point to Microsoft Visual Studio 2005, point to Visual Studio Tools, and click Visual Studio 2005 Command Prompt.
   
2. At the command prompt type a command in the following form, and press ENTER:
   

   gacutil.exe -i "<Full file system path to DLL>".

You can also use ".NET Framework 2.0 Configuration" tool to add the assembly to the GAC.

1. Go to control panel, click Administrative Tools and select Microsoft .NET Framework 2.0 Configuration.

2. Click Manage the Assembly Cache and then select Add an Assembly to the Assembly Cache. Locate the assembly and click Open.

Creating a Windows Application to Consume the Web Service

After copying the Web services files to the _vti_bin directory, the next step is to create a Windows Application to consume the Web service.

31. Create a new C# Windows Application in Visual Studio 2005.

32. In Solution Explorer, right-click References, and then click Add Web Reference.

33. In the address bar of the Add Web Reference browser, type the URL for the site to which to apply the service, as follows, and then press ENTER:

http://localhost/_vti_bin/Files.asmx?wsdl

If you installed the service on another port, for example, port 17316, then the url would be:
http://localhost:17316/_vti_bin/Files.asmx?wsdl
You can also browse all web services available on the server. Files.asmx will be listed in the web services available on the server. There will be multiple entries, each entry representing the service available for a different web application. Therefore, if you want to upload documents to a site on port 80, then you should select Files.asmx for port 80.
34. Include following using directives at the top:

using System.Net;
using System.IO;

35. Add following code to your application:

try
{
   localhost.Files oUploader = new localhost.Files();
   oUploader.PreAuthenticate = true;
   oUploader.Credentials = CredentialCache.DefaultCredentials;
   string strPath = @"C:\test.doc";
   string strFile = strPath.Substring(strPath.LastIndexOf("\\") + 1);
   string strDestination = "http://sp:17316/Docs";
   FileStream fStream = new FileStream(strPath, System.IO.FileMode.Open);
   byte[] binFile = new byte[(int)fStream.Length];
   fStream.Read(binFile, 0, (int)fStream.Length);
   fStream.Close();
   string str = oUploader.UploadDocument(strFile, binFile, strDestination);
   MessageBox.Show(str);
}
catch (Exception ex)
{
   MessageBox.Show(ex.Source + " - " + ex.Message + " - " + ex.InnerException + " - " + ex.StackTrace);
}

localhost is the name of the web reference that you added in previous step. strPath contains the filename that is to be uploaded. This is just a sample to show you how the service works, that's why i have hard coded the filename but in a real life situation you may want to add a text box and a browse button to select files from your hard disk. strDestination contains the destination path. This should be the path representing the document library where file is to be uploaded.
36. Finally, before testing the service, make sure the current user has privileges to upload documents in the destination document library. The user should have "Contributor" rights in the document library otherwise, you will get "401: Unauthorized" error. Run the application to test the service.

Check in multiple documents

You can simply just choose:

• Take ownership of documents you want to check in
• Site Actions
• Manage Content and Structure
• Find the folder where your files are
• Select all files (checkbox for checking all or single checkbox next to each file name)
• Choose Actions --> Check In
• Make a comment on the check in
• Done

Delete orphaned site collection

. Run the following command:

stsadm -o databaserepair -url http://<URL_of_WindowsSharePointServices_Site> -databasename <name of database that contains the orphan item>

2. If the above command reveals any orhpaned sites, then run the above command with -deletecorruption parameter:

stsadm -o databaserepair -url http://<URL_of_WindowsSharePointServices_Site> -databasename <name of database that contains the orphan item that is to be deleted> –deletecorruption

 

it should delete the site collection

if not check the content database associates with the orphaned site and delete the content data base

Sharepoint Import export

stsadm -o import -url http://servername/sites/gp -filename "c:\testweb" -includeusersecurity -nofilecompression -quiet

stsadm -o export -url http://server name/sites/gp -filename "c:\testweb" -includeusersecurity -versions 4 -nofilecompression –quiet

 

To move a sub site to root site of site collection

1. export the susite to c:\testweb folder

2. Create the root site using same site template

3.Import the c:\testweb file using import command

Incoming Email Enabled Libraries in MOSS2007 RTM using Exchange 2003 in an Active Directory Domain

How to configure Incoming Email Enabled Libraries in MOSS2007 RTM using Exchange 2003 in an Active Directory Domain.

This ‘How to’ guide will take you through installing and configuring your MOSS2007 environment for libraries to receive incoming e-mail using a standard e-mail address that can be used both internally and externally.
If you are interested in setting up incoming mail on Exchange 2007 then download my other whitepaper on the Combined Knowledge website http://www.combined-knowledge.com/Downloads%202007.htm
If you would like see some information on using Incoming E-Mail without using AD and Exchange refer to Todd Klindt’s post here http://www.toddklindt.com/blog/Lists/Posts/Post.aspx?ID=31
My server topology for this guide was as follows
Server Name
Role
Products Installed
Rootdc
Domain Controller
DNS Server
Exchange Server
Windows server 2003 SP1
Exchange Server 2003 SP1
SQL
Database server
SQL 2000 SP4
MOSS2007
MOSS2007 Server
MOSS2007 Enterprise
My internal domain name configured for the Active Directory is trainsbydave.com
You will need to replace trainsbydave.com with your own testing domain name
Troubleshooting – See the section at the end of the document
** updated April 2007 **
** updated July 2007 **
** updated November 2007 **
**updated May 2008 **
Step 1 – Let’s get the Active Directory Ready.
When you E-Mail enable a library in a site it will create a new contact in the OU we specify for the AD. Therefore it makes sense to have an OU dedicated to MOSS2007 list e-mail addresses.
On a Domain Controller Open Active Directory Users and Computers in the administration tools and create a new Organizational Unit under your main domain name. In my case I used the OU name sharepointdl in the domain trainsbydave.com
At this point we can new delegate control to the Central Administration Application Pool account to have Write access to the OU.
1. Right Click on the OU and click Delegate control
Add an MX record in DNS for the Moss Server.
Note – It is the account configured as the SharePoint Central Administration Application Pool in IIS that created the contact object in Active Directory, make sure this account his write access to the OU.
2. Click Next on the Wizard
3. Click Add to select the Central Admin Service Account from AD
4. Click Next
5. Tick ‘Create a Custom Task to Delegate’
6. Click Next
7. Accept the default setting for the next page
8. Click next
9. In the permissions box select Read, Write & Create all Child objects
10. Click Next
11. Click Finish
On the domain controller open DNS manager in the administrator tools. In DNS manager right click on your domain name and create a new MX record for your Moss server – moss2007, and point it to your moss server.
Ie. moss2007.trainsbydave.com
See Picture example on the next page
Step 2 – install the SMTP service on the MOSS 2007 Server
A) Add/Remove Programs, Windows Components, Application Server, IIS, SMTP Service
B) Configure SMTP Server to except relaying from the domain exchange server
a. Start, Administrative Tools, Internet Information Services
b. Right click on Properties of Default SMTP Virtual Server
c. Click the Access Tab, Relay. Add the IP address of the servers that can relay through this server. In my case this was the Exchange server in my domain
d. Click Ok, Ok, and Apply
OK so we now have an SMTP service running on the MOSS2007 server. We now need to configure the incoming e-mail settings on the MOSS 2007 server.
Step 3 – Configuring incoming e-mail settings in central administration
Open MOSS2007 Central Administration and click on the Operations Tab
Click Incoming E-Mail
On the incoming mail page we are now going to configure the page to use the Directory Management Service and configure this service to use the Organizational unit that we created earlier in order to create the new contact objects.
A) Click Yes to enable incoming mail and the settings mode on Automatic
B) In the Directory Management Service (DirMan) section select Yes to use the DirMan service.
1. In the Active Directory Field type the name of your Organizational unit that we created earlier. In my case this would be OU=sharepointdl,DC=trainsbydave,DC=com.
2. In the SMTP mail server field type the name of your moss server tat has the SMTP server installed. In my case this was the moss2007 server so the mail server address is moss2007.trainsbydave.com
3. Leave the default to Yes for Accepting messages from authenticated users only if you wish to use incoming mail for trusted senders only.
Note - If you are sending mail to the Moss Server from multiple locations and various account types then you may be need to enable No for incoming mail to work.
Allow Creation of distribution groups. If you want to allow Site Administrators to create E-Mail enabled distribution groups in Active Directory for their Site Groups then leave the default to yes and then select the approval setting for the DirMan service. By using approval the Site Administrator may be able to create a mail address for the Site Group but until a farm Administrator approves the new group it will not be created in AD.
To Approve the Site Groups mail enabled distribution group go the SharePoint Central Administration – Operations – Topology and Services – Approve/ Reject Distribution Groups.
If you select Advanced you are given the opportunity at the bottom of this page to specify which folder location will be used for the incoming mail drop folder. By default the drop folder will be the one used by the SMTP service which is located at C:\InetPub\Mailroot\Drop
Once mail is sent to this drop folder by the SMTP mail flow process then it is the responsibility of the SharePoint Timer Service to pick the mail up and distribute it to the correct List or Library.
Type the name of the mail domain that you wish to use with all new List or Libraries that are mail enabled in SharePoint sites associated with this SharePoint Farm. IE @moss2007.trainsbydave.com or @trainsbydave.com.
In my case I have used moss2007.trainsbydave.com which means when the contact object is created in Active Directory it will have a default SMTP alias of contact@moss2007.trainsbydave.com. However depending on how your mail routing is configured you may need to add the local SMTP address for the domain to the contact after the object is created is created in active Directory.
See later in this White Paper to see how to do this.
Finally choose which mail servers are allowed to route mail through this server. If you have a dedicated SMTP routing server in your organization then you could secure the mail traffic by specifying only the IP address here of your internal server. If you are not sure then the default to accept from all mail servers or speak with your mail server Admin.
When complete Click OK.
** Note ** If using a safe e-mail server you must use its IP address not an FQDN. If using an FQDN you may see the following error :
** If you receive any errors after clicking ok check the event logs on both the Domain controller and the Moss server to see what may be causing the problem. Most errors in this page are due to incorrect locations for the Organizational Unit specified in the Directory Management Service section. **
Step 4 – Creating a mail enabled document library in a team site
Now we have configured the Central administration side in MOSS 2007 we can create a new list and configure it to accept incoming e-mail. In my example I am going to create a document library and mail enable it to receive incoming mail.
I am going to call this document library fanmail as it will receive incoming mail messages sent to fanmail@moss2007.trainsbydave.com.
Browse to your team site and create a new library. – new document library
When creating the document library select ‘Yes’ for enable this document library to receive e-mail.
Add the e-mail address we are going to use in this case fanmail
Click Create
Set the email settings for the document library in the team site
Although we have mail enabled the library there are still a few settings that we need to decide on for the mail sent to this library.
1. Open the document library we have just created
2. Click Settings – document library settings
3. Under Communication Settings click E-Mail Settings
The settings in the library are all optional but for my test I am going to save all the attachments in the library plus I am going to keep a copy of the mail message in the library as well.
4. In the E-Mail Message Section Select Yes for save original e-mail
5. In the E-Mail Security section Select Yes to Accept e-mail messages from any sender
6. Click OK
If you choose to use the document library permissions for e-mail then only senders with add rights to the library can send mail to the library.
Step 5 – Managing the contacts created in Active Directory
Shortly after the document library has been created and mail enabled a new contact should appear automatically in the OU in Active Directory. This was the job of the directory management service to do this. (DIRMAN)
The default e-mail address will be fanmail@MOSS2007.trainsbydave.com as this is what we configured the mail addresses to be in MOSS2007 central administration – operations – incoming e-mail settings earlier in this document. This is fine for internal mail but if I wanted to have external fans from all over the world mail the library so I need to use a global company address. I.e.fanmail@trainsbydave.com.
I have two options to manage this.
Option 1 – Go into the properties of this contact and add a new SMTP mail address under the e-mail addresses tab for @trainsbydave.com. You will have to do this for each contact created
Option 2 – Create a new Exchange Server Global recipient policy so that all contacts created in that OU
Automatically get the second mail address added.
I have used Option 1 and edited the fanmail contact and added the @trainsbydave.com SMTP address. Now my contact can receive e-mail address to both SMTP addresses with the primary configured as fanmail@MOSS2007.trainsbydave.com
It may be necessary for some companies that you need to also add a new SMTP connector to your Exchange system in order to force all mail destined for the address space moss2007.trainsbydave.com to the IP address of the Moss2007 server that is hosting the local SMTP service that we installed and configured earlier. In my case this was not needed as I do not have a complex mail routing system.
Step 6 – Testing the solution
Let’s open Outlook now as a client and send a mail to fanmail address and follow its progress to appearing the document library
In my testing I am using Outlook web access rather than Outlook but the result is the same.
After sending the mail you can monitor the C:\Inetpub\mailroot\drop folder to see when WSS picks up the message.
Make sure that the SP Timer service is running on the MOSS2007 server as this server will be responsible for taking the mail from the Drop folder and putting it in the Document library.
After a short time the mail will appear in the drop folder configured earlier in Central administration – operations – incoming e-mail settings
Finally once the SPTimer service picks up the message it will disappear from the drop folder and the item will be created in the list.
Ensure that the SPTimer Service account can also delete objects from the chosen drop folder location. If it cannot then you may find that you get duplicate E-Mails.
Thanks to Chris Jones for this tip
Troubleshooting
Thanks for this list also go to Todd Klindt http://www.toddklindt.com/blog/default.aspx ( SharePoint MVP ) Mat Chase , Chris Jones , Ben Torrey and others in the community for providing some of these troubleshooting tips that helped them when they had problems in their own environments. This list does not mean you need to do all of them; they are suggestions to try in case you run into problems. All configurations are different and it is impossible for me to know how you have built your SharePoint up.
* Ensure that the Central Administration Application pool account and the SharePoint Timer Service account are using the same service account. THIS ONE IS IMPORTANT. A lot of errors stem from these accounts not being the same DOMAIN service account.
* Make sure the above account has access rights to all files on the server used by incoming mail such as the sharepointemailws.asmx file on the Moss server that is receiving the incoming mails. Alternatively make these accounts local administrators on the web servers receiving the incoming mail
* Make sure the app pool account for the web application you are trying to mail enable is running as the MOSS Administrator account you are running Central Administration under.
* Grant rights to the OU you have created in AD to the app pool account you setup.
* Create a SMTP send connector in Exchange to deliver mail to machine.domain.name where your MOSS SMTP service is running.
* Make sure the account that is running the SharePoint Timer Service has delete rights to the drop folder or you will receive duplicate E-Mails
* When using Outlook 2003 may need to send the attachment as a Uuencode or Binhex format in order to get the attachment listed separately in the List or Library. Also see this MS article for another workaround for adding two attributes to the contact in AD
http://support.microsoft.com/default.aspx?scid=kb;en-us;926891
* If you have installed Forefront security for SharePoint then you may encounter a problem where the E-Mail will arrive in the drop folder and then disappear without reaching the document library. This MS article explains the way to fix this. http://support.microsoft.com/kb/934285
* If you have used permissions on the document library to control who can send the process is based on the header of the E-Mail , it is then checked against the users who have rights to the list or library.
* If you see content in the document library that is assigned to the system account it means that the document library has treated the incoming mail as an anonymous users due to not be able to resolve the name in the list of users with rights to submit mail. You will have to allow the library to accept mail from all senders to see this.
* When sending to a calendar list to be sent correctly send as a .vcf file from a new appointment or meeting request in Outlook
Troubleshooting - continued
* If you try and setup a workflow on the mail as it arrives in the library and it fails make sure that the Timer Service and the Central Admin Application pool account are using the same Domain service account.
* When supporting multiple SMTP domains make sure that you add these domains to the SMTP domains in the SMTP settings on the Moss Server hosting the SMTP service. SharePoint can support external domain names as long as the domains are supported by the mail routing topology and also the Moss servers SMTP domain name listing.
I hope you found this article useful , please do feel free to send me comments and keep checking our web site for more downloads on configuring some of the cool new features of Moss2007.
Steve@combined-knowledge.com

Incoming Email Enabled Libraries in MOSS2007 RTM using Exchange 2007 in an Active Directory Domain

How to configure Incoming Email Enabled Libraries in MOSS2007 RTM using Exchange 2007 in an Active Directory Domain.

This ‘How to’ guide will take you through installing and configuring your MOSS2007 environment for libraries to receive incoming e-mail using an e-mail address using Exchange 2007.
My server topology for this guide was as follows
Server Name
Role
Products Installed
DC
Domain Controller
DNS Server
Windows server 2003 SP1
MAIL
Mail Server
Exchange 2007 Enterprise
SQL
Database server
SQL 2005 SP1
MOSS2007
MOSS2007 Server
MOSS2007 Enterprise
My internal domain name configured for the Active Directory is trainsbydave.com
You will need to replace trainsbydave.com with your own testing domain name
Troubleshooting – See the section at the end of the document
** updated April 2007 **
** updated July 2007 **
** updated November 2007 **
**updated May 2008 **
Step 1 – Let’s get the Active Directory Ready.
When you E-Mail enable a library in a site it will create a new contact in the OU we specify for the AD. Therefore it makes sense to have an OU dedicated to MOSS2007 list e-mail addresses.
On a Domain Controller Open Active Directory Users and Computers in the administration tools and create a new Organizational Unit under your main domain name. In my case I used the OU name sharepointdl in the domain trainsbydave.com
At this point we can new delegate control to the Central Administration Application Pool account to have Write access to the OU.
1. Right Click on the OU and click Delegate control
Note – It is the account configured as the SharePoint Central Administration Application Pool in IIS that created the contact object in Active Directory, make sure this account his write access to the OU.
1 – It is the account configured as the SharePoint Central Administration Application Pool in IIS that created the contact object in Active Directory, make sure this account his write access to the OU.
Note 2 – Even though the objects are going to be created in the active directory they will now be managed in the Exchange management console. This is a significant change from Exchange 2003 where objects were managed in AD only. If you are using Exchange 2003 please refer to my previous white paper on configuring incoming e-mail with Exchange 2003 which can be found here – http://www.combined-knowledge.com/downloads%202007.htm
2. Click Next on the Wizard
3. Click Add to select the Central Admin Service Account from AD
4. Click Next
5. Tick ‘Create a Custom Task to Delegate’
6. Click Next
7. Accept the default setting for the next page
8. Click next
9. In the permissions box select Read, Write & Create All Child Objects
10. Click Next
11. Click Finish
Add an MX record in DNS for the Moss Server.
On the domain controller open DNS manager in the administrator tools. In DNS manager right click on your domain name and create a new MX record for your Moss server – moss2007, and point it to your moss server.
Ie. moss2007.trainsbydave.com
See Picture example on the next page
** The reason why we must have this DNS entry is to ensure that we can resolve the e-mail domain name to the Moss server once the E-mail is received by your mail server. What happens is that the mail is received by the Exchange 2007 mail system. The mail system now needs to know where to send the mail with an address of xyz@moss.trainsbydave.com. It will do a lookup on the directory of mail addresses used by Exchange 2007 in order to resolve the name. Once it has found the matching e-mail address for the object it will either send the mail to the object held on the Exchange server such as a mailbox or it will forward to another mail server if configured in a SMTP send connector that matches the address space.
In our case we will need to make sure that the incoming mail for Moss lists is forwarded to the Moss server that has the SMTP service installed and for that we will need a DNS entry as well as an SMTP send connector in Exchange. We will cover creating an SMTP send connector later in this whitepaper.
Step 2 – install the SMTP service on the MOSS 2007 Server
A) Add/Remove Programs, Windows Components, Application Server, IIS, SMTP Service
B) Configure SMTP Server to except relaying from the domain exchange server
a. Start, Administrative Tools, Internet Information Services
b. Right click on Properties of Default SMTP Virtual Server
c. Click the Access Tab, Relay. Add the IP address of the servers that can relay through this server. In my case this was the IP address of the Exchange server in my domain
d. Click Ok, Ok, and Apply
OK so we now have an SMTP service running on the MOSS2007 server. We now need to configure the incoming e-mail settings on the MOSS 2007 server.
Step 2 – Configuring incoming e-mail settings in central administration
Open MOSS2007 Central Administration and click on the Operations Tab
Click Incoming E-Mail
On the incoming mail page we are now going to configure the page to use the Directory Management Service and configure this service to use the Organizational unit that we created earlier in order to create the new contact objects.
A) Click Yes to enable incoming mail and the settings mode on Automatic
B) In the Directory Management Service (DirMan) section select Yes to use the DirMan service.
1. In the Active Directory Field type the name of your Organizational unit that we created earlier. In my case this would be OU=sharepointdl,DC=trainsbydave,DC=com.
2. In the SMTP mail server field type the name of your moss server tat has the SMTP server installed. In my case this was the moss2007 server so the mail server address is moss2007.trainsbydave.com
3. Leave the default to Yes for Accepting messages from authenticated users only if you wish to use incoming mail for trusted senders only.
Note - If you are sending mail to the Moss Server from multiple locations and various account types then you may be need to enable No for incoming mail to work.
Allow Creation of distribution groups. If you want to allow Site Administrators to create E-Mail enabled distribution groups in Active Directory for their Site Groups then leave the default to yes and then select the approval setting for the DirMan service. By using approval the Site Administrator may be able to create a mail address for the Site Group but until a farm Administrator approves the new group it will not be created in AD.
To Approve the Site Groups mail enabled distribution group go the SharePoint Central Administration – Operations – Topology and Services – Approve/ Reject Distribution Groups.
If you select Advanced you are given the opportunity at the bottom of this page to specify which folder location will be used for the incoming mail drop folder. By default the drop folder will be the one used by the SMTP service which is located at C:\InetPub\Mailroot\Drop
Once mail is sent to this drop folder by the SMTP mail flow process then it is the responsibility of the SharePoint Timer Service to pick the mail up and distribute it to the correct List or Library.
Type the name of the mail domain that you wish to use with all new List or Libraries that are mail enabled in SharePoint sites associated with this SharePoint Farm. IE @moss2007.trainsbydave.com or @trainsbydave.com.
In my case I have used moss2007.trainsbydave.com which means when the contact object is created in Active Directory it will have a default SMTP alias of contact@moss2007.trainsbydave.com. However depending on how your mail routing is configured you may need to add the local SMTP address for the domain to the contact after the object is created is created in active Directory.
See later in this White Paper to see how to do this.
Finally choose which mail servers are allowed to route mail through this server. If you have a dedicated SMTP routing server in your organization then you could secure the mail traffic by specifying only the IP address here of your internal server. If you are not sure then the default to accept from all mail servers or speak with your mail server Admin.
Then Click OK
** You must use an IP address if using safe e-mail servers. FQDN names will give you the following error.
If after clicking OK you receive ‘ERROR IN APPLICATION’
This is usually caused by a permissions error. Check the troubleshooting section at the end of the white paper for some suggested things to check. You may also need to check the logs of your moss servers, active directory and exchange though to try and track down the exact cause of the permissions error.
You may also get an error due to an incorrect location or typed mistake for the Organizational Unit specified in the Directory Management Service section.
Step 3 – Creating a mail enabled document library in a team site
Now we have configured the Central administration side in MOSS 2007 we can create a new list and configure it to accept incoming e-mail. In my example I am going to create a document library and mail enable it to receive incoming mail.
I am going to call this document library fanmail as it will receive incoming mail messages sent to fanmail@moss2007.trainsbydave.com.
Browse to your team site and create a new library. – new document library
When creating the document library select ‘Yes’ for enable this document library to receive e-mail.
Add the e-mail address we are going to use in this case fanmail
Click Create
Set the email settings for the document library in the team site
Although we have mail enabled the library there are still a few settings that we need to decide on for the mail sent to this library.
1. Open the document library we have just created
2. Click Settings – document library settings
3. Under Communication Settings click E-Mail Settings
The settings in the library are all optional but for my test I am going to save all the attachments in the library plus I am going to keep a copy of the mail message in the library as well.
4. In the E-Mail Message Section Select Yes for save original e-mail
5. In the E-Mail Security section Select Yes to Accept e-mail messages from any sender
6. Click OK
If you choose to use the document library permissions for e-mail then only senders with add rights to the library can send mail to the library.
Step 4 – Managing the contacts created in Active Directory
Shortly after the document library has been created and mail enabled a new contact should appear automatically in the OU in Active Directory. This was the job of the directory management service to do this. (DIRMAN)
Although the contact is in Active Directory we now need to go and manage the contact in Exchange 2007 and we also need to create the SMTP send routing connector in Exchange as well.
** In my testing environments you must always create a Send connector to the Moss server SMTP address space in order for mail routing to work correctly from Exchange 2007 ***
Go to your Exchange 2007 Server and open the Exchange Management Console.
Click on Hub Management Transport
Click on the Send Connectors Tab
In the right hand Actions column click ‘New send Connector’
Give your new connector and name and select the appropriate untended use. In my case this was for internal connection use.
Click Next.
We now need to add an Address space to tell the Exchange server about the Moss server. This address space will be used by Exchange to route any mail received for the moss server addresses directly to the SMTP service running on the Moss server configured for incoming mail.
Click add and complete the name to the Moss server configured to receive incoming mail.
Click OK to add the address Space entry
Then Click Next
** If you want to support multiple mail addresses on your Moss server then you can add multiple address spaces here. **
Now we need to add the IP address or Fully Qualified domain name of the smart host to receive this address space. This will be the IP address of the Moss server.
Add the address and Click OK
Then click Next
If you wish to further secure communication between Exchange and the Moss SMTP server then you can configure additional security. By default you should select ‘None’
On the next screen you can choose an additional Hub transport server if you have more than one Exchange server handling mail routing. In our case it is the default Exchange server.
Click Next
Check the Summary page and then click ‘New’ to continue.
Click Finish.
Let’s now look at the contact created in the Exchange management console Go to Recipient Configuration Click mail Contact Find the SharePoint library contact object select it and click properties Then click E-mail addresses
** Make sure that the SMTP E-Mail address for the contact is present. If not or you get an error when clicking properties make sure that the recipient policy has been applied. You can force the recipient policy to run by running the recipient policy wizard in the Exchange management console as part of the organizational configuration - hub transport settings. Alternatively waiting for a short time will usually update automatically. **
Testing the solution
Let’s open Outlook now as a client and send a mail to fanmail address and follow its progress to appearing the document library
In my testing I am using Outlook web access rather than Outlook but the result is the same.
After sending the mail you can monitor the C:\Inetpub\mailroot\drop folder to see when WSS picks up the message.
Make sure that the SP Timer service is running on the MOSS2007 server as this server will be responsible for taking the mail from the Drop folder and putting it in the Document library.
After a short time the mail will appear in the drop folder configured earlier in Central administration – operations – incoming e-mail settings
Finally once the SPTimer service picks up the message it will disappear from the drop folder and the item will be created in the list.
Ensure that the SPTimer Service account can also delete objects from the chosen drop folder location. If it cannot then you may find that you get duplicate E-Mails.
Thanks to Chris Jones for this tip
Troubleshooting
Thanks for this list also go to Todd Klindt http://www.toddklindt.com/blog/default.aspx ( SharePoint MVP ) Mat Chase , Chris Jones , Ben Torrey and others in the community for providing some of these troubleshooting tips that helped them when they had problems in their own environments. This list does not mean you need to do all of them; they are suggestions to try in case you run into problems. All configurations are different and it is impossible for me to know how you have built your SharePoint up.
* Ensure that the Central Administration Application pool account and the SharePoint Timer Service account are using the same service account. THIS ONE IS IMPORTANT. A lot of errors stem from these accounts not being the same DOMAIN service account.
* Make sure the above account has access rights to all files on the server used by incoming mail such as the sharepointemailws.asmx file on the Moss server that is receiving the incoming mails. Alternatively make these accounts local administrators on the web servers receiving the incoming mail
* Make sure the app pool account for the web application you are trying to mail enable is running as the MOSS Administrator account you are running Central Administration under.
* Grant rights to the OU you have created in AD to the app pool account you setup.
* Create a SMTP send connector in Exchange to deliver mail to machine.domain.name where your MOSS SMTP service is running.
* Make sure the account that is running the SharePoint Timer Service has delete rights to the drop folder or you will receive duplicate E-Mails
* When using Outlook 2003 may need to send the attachment as a Uuencode or Binhex format in order to get the attachment listed separately in the List or Library. Also see this MS article for another workaround for adding two attributes to the contact in AD
http://support.microsoft.com/default.aspx?scid=kb;en-us;926891
* If you have installed Forefront security for SharePoint then you may encounter a problem where the E-Mail will arrive in the drop folder and then disappear without reaching the document library. This MS article explains the way to fix this. http://support.microsoft.com/kb/934285
* If you have used permissions on the document library to control who can send the process is based on the header of the E-Mail , it is then checked against the users who have rights to the list or library.
* If you see content in the document library that is assigned to the system account it means that the document library has treated the incoming mail as an anonymous users due to not be able to resolve the name in the list of users with rights to submit mail. You will have to allow the library to accept mail from all senders to see this.
* When sending to a calendar list to be sent correctly send as a .vcf file from a new appointment or meeting request in Outlook
Troubleshooting - continued
* If you try and setup a workflow on the mail as it arrives in the library and it fails make sure that the Timer Service and the Central Admin Application pool account are using the same Domain service account.
* When supporting multiple SMTP domains make sure that you add these domains to the SMTP domains in the SMTP settings on the Moss Server hosting the SMTP service. SharePoint can support external domain names as long as the domains are supported by the mail routing topology and also the Moss servers SMTP domain name listing.
I hope you found this article useful , please do feel free to send me comments and keep checking our web site for more downloads on configuring some of the cool new features of Moss2007.
Steve@combined-knowledge.com

SharePoint Incoming email settings

SharePoint Incoming email

1.Email comes to drop folder but does not redirect to appropriate list or library.

Go to central admin--> operations-->Timer Job definitions

Check whether the WS SharePoint incoming email job is running
if it is not running

1)  ran STSADM -o enumservices > c:\services.txt   to list the services in a text file.
Then they opened the text file and found the SPIncomingEmailService and copied that entry.
Then they ran STSADM -o provisionservice -action start -servicetype "Microsoft.SharePoint.SPIncomingEmailService, Microsoft.SharePoint, Version=12.0.0.0, Culture..."

      Make sure the string for the service type above is in quotations and that you copy the entire string from the services txt file...above I abbreviated it but you need the entire string.

That made the service start up immediately.

2) Then I had to go into Central Administration - Operations - Incoming Email settings and change my from Automatic to Advanced and point to the proper Drop folder.  I should have to do that, but in my case I had just removed a second front end server out of the farm and SharePoint kept thinking SMTP should be on the other server that was decommissioned.

restart windows SharePoint services timer

3) Then an IISreset /noforce  and everything started sending.

check whether the WS SharePoint incoming email job is running

then it should work

link for SharePoint incoming email settings

http://www.combined-knowledge.com/Downloads_2007.html