SAS Community Planet

The Princess of Science: ‫מהלימון ועד הקופקסון – מצגת‬

Thu, 18 Mar 2010 11:53:37 +0000

‫היום נתתי במועדון קשישים בתל-אביב הרצאה שנשאה את הכותרת: "מהלימון ועד הקופקסון – קיצור תולדות הנסויים הקליניים", לחיצה על הקישור תפתח קובץ pdf של מצגת ההרצאה. אני מקווה לכתוב אחלק מהדברים בצורת רשימה מסודרת כאן בבלוג בעתיד הקרוב. חלק מהדברים כבר מוכרים לכם, ומבוססים על הרשימה "הסטטיסטיקה שהצילה חיים – סיפורה של פלורנס נייטינגייל" [...]‬

NOTE: The blog of RTSL.eu: Project Plans in Excel - As a Chart

Wed, 17 Mar 2010 19:20:07 +0000

The series of posts on project planning in Excel with Gantt charts has been very popular, and one of the most popular questions has been "Why don't you use an Excel chart?" Well, the answer's simple, they don't work very well for large list of tasks. For small lists they look very nice, but they don't scale well, hence I prefer to keep my Gantt in the cells of the worksheet. However, for completeness I thought I'd offer this bonus post to show how it's done. You can see the end result alongside this paragraph (right). I'm using Excel 2003.

We'll start with the result from the last post, including the progress bars in the worksheet. You can see it alongside this paragraph (left). Since we used the cells to indicate progress, we were limited to showing progress in chunks of whole days. In the chart we will be able to show a more accurate picture of progress.

I'm going to start by removing the groups that we had in the last result - I've never explored how they can successfully be charted. So, let's select the input data area (A2 to F9), go to the Subtotals window, and click the Remove All button (then confirm that you understand that entire rows will be removed). Our chart collapses and looks as shown below.

Let's add a couple of columns that indicate the amount of progress (equivalent to column Q) and the amount remaining. So, insert a couple of columns between Start and Effort. To keep this tutorial simple, the position is important, but you could actually put these columns over to the right if you're familiar with de-selecting data series in Excel charts. Having inserted two columns, let's label them Done and Remain, and let's set their values with formulae. Put D3 =F3*H3/100, and E3 =F3-D3. In other words, Done equals Effort times %Done, and Remain equals Effort minus Done. Copy these formulae down and you'll get the result below.

Now select the area that contains the data to be charted, i.e. B2 to E6, and launch the Chart wizard. In step1 of the Wizard, select chart type of Bar and sub-type of Stacked Bar (see below).

Click Finish. Let's see what we've got:

We can see that the order down the left side is wrong (activity #4 is above activity #3, we'd like activity #1 at the top); if we hide the purple Start bars then we'll have some nice bars that show Done and Remain.

Right-click on the left axis, choose Format Axis, go to the Scale tab and tick Categories in reverse order. See below.

Click OK and you'll see that the order of tasks has reversed. Now right click on any of those purple Start bars, choose Format Data Series, and on the Patterns tab select Border colour of None and Area colour of None. Don't press OK yet. See below.

Then, before you click OK, go to the Data Labels tab and put a tick alongside Category Name. Now you can click OK and you'll see that the purple Start bars have disappeared.

Get rid of the Legend by right clicking on it and selecting Clear. And get rid of the left axis labels by right clicking on one of them and selecting Clear.

It's starting to look like what we want. Right click on one of the activity labels and select Format Data Labels. On the Alignment tab, select Right for Horizontal alignment, and Inside End for Label Position. See below.

Click OK. The labels look neat (although the labels for activity #1 and #3 unavoidably overlap the bars because of the limitations of the left margin).

Right, let's make it look a bit more snappy. Click on the grey background and select Clear; click on the top axis, select Format Axis, and select Lines as None in the Patterns tab; click on the border of the chart and select Format Chart Area, select Round Corners on the Patterns tab, and also select Fill Effects for the Area and then choose one colour and apply a vertical shading style. After having set the fonts to bold, the grid-lines to grey, and bar colours to red and green, here's my result:

They say beauty is in the eye of the beholder, so you may think it looks snappy, or you may not. Either way, it's an eye-catching Gantt chart.

As I said at the top, I find these charts creak under the strain of a larger number of tasks, and they become troublesome to print, hence I prefer using the cells of the worksheet.

NOTE: The blog of RTSL.eu: Project Plans in Excel - Tracking to Completion

Wed, 17 Mar 2010 19:18:42 +0000

The series on maintaining a project plan and Gantt chart in Excel has been popular, and I've had a lot of queries about tracking progress. So, in this bonus post I’ll describe how to display tasks’ progress on the Gantt chart that was featured in the previous posts in this series. In addition, I’ll show how to highlight “today”. Alongside this paragraph (right) you can see what the result of this post looks like.

In the three previous posts in this series I described how to create a neat and simple Gantt chart, how to add dates to the day numbers, and how to group tasks. These three simple sets of steps have given the developer sufficient knowledge to quickly create a simple but effective Gantt chart that demonstrates the developer is in control of the project (without spending more time on planning than on delivery). Alongside this paragraph (left) you can see what the results of our previous efforts looks like.

Let’s start by inserting a column after E and heading it “%Done”. This is where you'll need to type values to indicate your progress. Then, after column N let’s add “Done” and “DoneEnd”. I’ve made the text colour of the latter two columns a semi-visible grey because they’re our working values and not of interest to the reader of the Gantt chart. If you see the picture below, you’ll see that I’ve also populated the %Done column with some values.

Now let’s populate the calculated columns. The working columns are not strictly necessary, but they’ll help illustrate the calculations that we’re doing. Firstly, let’s understand what we’re trying to achieve. Cell H3 represents progress on activity #1 on day 1. We’ll display a block in the cell if progress on the activity is equal to (or greater than) half a day’s effort. So, for activity #1 we can see that effort is complete up to the end of day 3; for activity #2 the effort is complete up to 2/3 of the way through day number 2. Since day number 2 for activity #2 is more than half complete we’ll put a block in that cell (but not day number 3).

Let’s set Done to show the amount of effort completed, i.e. %Done times Effort (so Q3 =D3*F3/100). We’ll use DoneEnd to indicate the cell location where the completed effort ends (so R3 =C3-1+(F3*D3/100)).

DoneEnd=1.6 in cell R4 tells us that the effort is complete up to 0.6 of the way through the cell that represents day 2. So we’d place in block in day 1 and we’d also place a block in day 2 because it’s more than half complete.

We’ve previously used conditional formatting to indicate the start and end dates, so we’ll use formulae to shows blocks for completed effort in appropriate cells. The formula for cell H3 is =IF(AND($C3<=H$2,H$2-0.5<=$R3),"n",""). Once we’ve spread this across the whole of the Gantt we get the following result.

Oh, and change the font for the Gantt area to Wingdings so the blocks appear as blocks rather than lower-case n characters! You can also change the text colour to something other than black.

So, we now have %Done bars (blocks) overlaying our basic start-to-end Gantt bars. But we’ve avoided doing this for the grouped bars. Why? Well, we can calculate %Done as the sum of Done for the group divided by total effort for the group, i.e. F5 =100*Q5/D5 (first, you need to set Q5 to =SUBTOTAL(9,Q3:Q4)), but if you add this as blocks in the Gantt you’ll see that the bars run from day 1 to day 4 despite the fact that the individual activities for group Alpha have not hit day 4 yet. This can be confusing for the reader of the Gantt, so I usually keep the calculation, but remove the progress bars, leaving the end result below.

In addition to copying D5 to Q5, I’ve deleted R5 (just for clarity, because we’re not using it). And I hope you realised you need to do similar things in the other grouped rows, so Q8 contains =SUBTOTAL(9,Q6:Q7), and Q9 contains =SUBTOTAL(9,Q3:Q7).

Finally, I said I’d highlight “today”. Well that's easy, select the date cells (H1 to O1), open-up Conditional Formatting and set the formula =IF(H1=TODAY(),1,0). I’ve chosen to use white text on a black background:

Here’s the final spreadsheet:

Neat!

SAS from Out in Left Field: Excel Pivot Tables and Filtering

Wed, 17 Mar 2010 13:40:39 +0000

So I get a post this morning from my client saying everything looks great and they can't find any issues with the latest release. Happy moment, cigar time.

Well, at this point I am over on the project and am working on my own time. That's fine considering that there might be future business there. Ok, so they mention 2 minor items that are annoyances that would be perfect. No obligation to do them but I have a few minutes so I dive in.

5 hours later I have it working. What was the issue? Pivot Tables in Excel and setting their filter values. For those poor saps who have to deal with it in the future, here is some C# code to hopefully save you some time:


        private static void HandlePivotTableChanges(string p)
        {
            FindProcessInstances();
            Console.WriteLine("Starting Excel. Please be patient while it loads.");
            string format = "'Product_Container_Release_Metrics_Workbook.xls'!Format_Workbook";
            Application ExcelApp = new Microsoft.Office.Interop.Excel.Application();
            ExcelApp.DisplayAlerts = false;
            Console.WriteLine("Opening workbook.");
            Workbook wb = ExcelApp.Workbooks.Open(p, m, m, m, m, m, m, m, m, m, m, m, m, m, m);
            Console.WriteLine("Running macro.");
            ExcelApp.RunMacro(format);
            Worksheet ws = (Worksheet)wb.Worksheets["Pivot Tables"];
            string pivotTable = "Forecast to Go";
            PivotTable pt = (PivotTable)ws.PivotTables(pivotTable);
            pt.ManualUpdate = true;
            PivotField pf = (PivotField)pt.PivotFields("Release (A)");
            PivotItems items = (PivotItems)pf.PivotItems(m);

            foreach (PivotItem item in items)
            {
                    if (item.Value != string.Empty)  // --- CRITICAL LINE. Filters must have at least 1 value selected. Exception otherwise.
                        ((PivotItem) items.Item(item.Value)).Visible = false;
            }

            pt.ManualUpdate = false;
            ExcelApp.Calculation = XlCalculation.xlCalculationAutomatic;
            Console.WriteLine("Saving workbook.");
            wb.Save();
            wb.Close(false, m, m);
            System.Runtime.InteropServices.Marshal.FinalReleaseComObject(wb);
            Console.WriteLine("Closing Excel.");
            ExcelApp.Quit();
            System.Runtime.InteropServices.Marshal.FinalReleaseComObject(ExcelApp);
        }

        private static void FindProcessInstances()
        {
            Console.WriteLine("Finding open processes.");
            KillProcesses("Excel");
            KillProcesses("PowerPnt");
        }

        private static void KillProcesses(string processName)
        {
            Process[] processes = Process.GetProcessesByName(processName);
            if (processes.Length > 0)
            {
                foreach (var p in processes)
                {
                    p.Kill();
                }
            }
        }

        public static void RunMacro(this Application app, string macro)
        {
            app.Run(macro,
                    m, m, m, m, m, m, m, m, m, m,
                    m, m, m, m, m, m, m, m, m, m,
                    m, m, m, m, m, m, m, m, m, m
                );
        }

StudySAS Blog: PRXMATCH Function

Wed, 17 Mar 2010 09:31:06 +0000

Prxmatch () function is very useful in locating the matching strings. Prxmatch() function has 2 parameters, the first parameter is the regular expression ID (i.e what you are looking in a string for a match) and the second parameter is the character string to be searched. PRXMATCH () function returns the start position of the matching string. Syntax: PRXMATCH (perl-regular-expression, source); Even though PRXMATCH function can be used when.... 1) When you want to identify if there is alphanumeric (has any letter from A to Z) in a variable. 2) If you need to search a character variable for multiple different substrings. Here is how PRXMATCH works in the Ist case. *Prxmatch () function is very useful in locating the matching strings; DATA finda2z; INPUT ID $ 1-3 string $ 5-10; prxmatch=prxmatch("/[a-zA-Z]/",string); DATALINES; 001 ACBED 002 11 003 12 004 zx 005 11 2c 006 abc123 ; run; proc print; run; Output: *Here...

[[ This is a content summary only. Visit my website for full links, other content, and more! ]]

The SAS Dummy: Analytics: getting sportier all of the time

Tue, 16 Mar 2010 14:32:27 +0000

It's March, and in North Carolina all of the talk is about college basketball. At SAS, we like to brag about how these guys use SAS analytics to predict the NCAA "dance card" with astounding results. This year, the fun question revolves around conference bias and how it affects a team's chances.

The use of analytics has evolved as an important sports discipline in recent years. It's not just baseball, a sport known for its statistics. Anything that can be quantified or categorized can be analyzed for influence, including the birth month of professional soccer players.

My friend Drew Cannon, a high school student himself just a couple of years ago, applies analytical research to the business of recruiting high-school basketball players onto college teams. Drew's efforts are highlighted in a New York Times article today.

NOTE: The blog of RTSL.eu: NOTE: "SAS Talks" Has a Home

Tue, 16 Mar 2010 13:15:00 +0000

The excellent SAS Talks series of free webinars (previously featured in NOTE:) has now got a home page where you can find previously recorded talks, plus information about upcoming talks.

So, you can now easily catch-up with the previous talks on SAS Enterprise Guide, SAS/IML, ODS statistical graphics, and multi-sheet output for Microsoft Excel from SAS.

The page shows two upcoming talks: Modifying ODS Statistical Graphics Templates in V9.2 later in March, and Formats and the Format Procedure in April. Registration for the former opens today. If you watch it live you can question the speaker.

The Princess of Science: ‫הודעה מנהלתית על חידון פיי‬

Tue, 16 Mar 2010 09:28:33 +0000

‫קיבלתי כמה תגובות (בבלוג ובמייל) בהן נתבקשתי לפרסם את התשובות לחידון הפיי שפרסמתי בתחילת השבוע. אני מציע שהקוראים ינסו את כוחם ויכתבו את תשובותיהם בתגובות לחידון. אפרסם את התשובות מתישהו בשבוע הבא. לקריאה נוספת בנושאים הקשורים לנושא רשימה זו אין רשימות קשורות . ‬

SAS Programming: An efficient macro for Stump - two terminal nodes tree

Tue, 16 Mar 2010 01:34:18 +0000

In this post, I post an improved SAS macro of the single partition split algorithm in Chapter 2 of "Pharmaceutical Statistics Using SAS: A Practical Guide" by Alex Dmitrienko, Christy Chuang-Stein, Ralph B. D'Agostino.

The single partition split algorithm is a simplified version of Stumps, and is a weak classifier, usually used to form the base weak learner for boosting algorithm. This specific classifier seeks to separate the space into 2 subspaces for independent variables where each subspace has increasingly higher purity of the response classes, say 1 and 0. In the examples, Gini Index is used to measure purity/impurity.

The SAS example macro %SPLIT in the book (found @ Here) is for illustration purpose, and is so inefficient that it practically can't be used by industrial standard.

I modified this macro and made it usable in real business applications where millions of observations and hundreds of variables are more than common.

Improved Code:



%macro Dsplit(dsn, p);
/************************************************/
/* dsn: Name of input SAS data sets. All        */
/*        independent variables should be named */
/*        as X1, X2,....,Xp and be continous    */
/*        numeric variables                     */
/*   p: Number of independent variables         */
/************************************************/
options nonotes;
%do i=1 %to &p;
proc sort data=&dsn.(keep=y w x&i)  out=work.sort; by x&i;
run;

proc means data=work.sort noprint;
     var y;
     weight w;
     output out=_ysum  n(y)=ntotal  sum(y)=ysum;
run;
data _null_;
     set _ysum;
     call symput('ntotal', ntotal);
     call symput('ysum', ysum); 
run;
data y_pred;
     set work.sort  end=eof;
     array _p[&ntotal, 2] _temporary_;
     array _g[&ntotal, 2] _temporary_;
     array _x[&ntotal]    _temporary_;
     retain _y1  _oldx 0;
     if _n_=1 then _oldx=x&i;
     _x[_n_]=x&i;
     if ^eof then do;
        _y1+y; 
       _p[_n_, 1]=_y1/_n_; _p[_n_, 2]=(&ysum-_y1)/(&ntotal-_n_);
       ppn1=_n_/&ntotal;  ppn2=1-ppn1;
       _g[_n_, 1]=2*(ppn1*(1-_p[_n_, 1])*_p[_n_, 1]+ppn2*(1-_p[_n_, 2])*_p[_n_, 2]);
       if _n_>1 then _g[_n_-1, 2]=(_oldx+x&i)/2;
       _oldx=x&i;
     end;
     else do;
       _g[_n_-1, 2]=(_oldx+x&i)/2;
       ginimin=2;
       do i=1 to &ntotal-1;
          gini=_g[i, 1]; x&i=_g[i, 2];
          keep gini x&i;
          output;
     if gini lt ginimin then do;
        ginimin=gini; xmin=x&i;
            p1_LH=_P[i, 1]; p0_LH=1-p1_LH;
              p1_RH=_P[i, 2]; p0_RH=1-p1_RH;
        c_L=(p1_LH>0.5); c_R=(p1_RH>0.5);
    end;
       end;
     end;  
     do i=1 to &ntotal;
        if _x[i]<=xmin then y_pred=c_L;
        if _x[i]>xmin  then y_pred=c_R;
        keep y_pred y w; output y_pred;
     end;
     call symput('ginimin', ginimin);
     call symput('xmin', xmin);
     end;
run;
data _giniout&i;
     length varname $ 8;
     varname="x&i";
     cutoff=&xmin;
     gini=&ginimin;
run;
%end;
data outsplit;
     set %do i=1 %to &p;
            _giniout&i
         %end;;
run;
proc datasets library=work nolist;
     delete _giniout:;
quit;
option notes;    
%mend;

/* pure stump rules */

%macro stump(dsn, p);
/************************************************/
/* dsn: Name of input SAS data sets. All        */
/*        independent variables should be named */
/*        as X1, X2,....,Xp and be continous    */
/*        numeric variables                     */
/*   p: Number of independent variables         */
/************************************************/
options nonotes;
%do i=1 %to &p;
proc sort data=&dsn.(keep=y w x&i)  out=work.sort; by x&i; run;

data _null_;
     set work.sort  end=eof  nobs=ntotal;
  retain _wsum _ysum;
  _wsum+w; _ysum+y*w;
  if eof then do;
     factor=ntotal/_wsum;
     call symput('ysum', compress(_ysum*factor));
  call symput('ntotal', compress(ntotal));
  call symput('factor', compress(factor));
  end;
run;

%put factor=&factor;
%put ysum=&ysum;

data _giniout&i;
     array _p[2] _temporary_;
     array _g[2] _temporary_;
     array _x[1] _temporary_;
     retain ginimin   xmin  _y1  _oldx _cn ;
     if _n_=1 then do;
        ginimin=2; xmin=x&i; _y1=0; _oldx=0; _cn=0;
     end;    
     do while (^eof);
        set work.sort  end=eof;
        _y1+y*w*&factor; _cn=_cn+w*&factor;
        if _cn=0 then _p[1]=0; else _p[1]=_y1/_cn; 
        if _cn=&ntotal then _p[2]=0; else _p[2]=(&ysum-_y1)/(&ntotal-_cn);
        ppn1=_cn/&ntotal;  ppn2=1-ppn1;
        _g[1]=2*(ppn1*(1-_p[1])*_p[1]+ppn2*(1-_p[2])*_p[2]);
        if _n_=1 then _g[2]=x&i; else _g[2]=(_oldx+x&i)/2;
        _oldx=x&i;
        if _g[1]2 then do; 
        ginimin=2; xmin=(_oldx+x&i)/2; 
     end; 
     varname="x&i";
     keep varname  ginimin  xmin;
     output;
     stop;
run;
%end;
data outsplit;
     set %do i=1 %to &p;
            _giniout&i
         %end;;
run;
proc datasets library=work nolist;
     delete _giniout:;
quit;
option notes;    
%mend;

Comparing the time used and results.

The example data used is the AUC Small training data, 200 sets of predictors for 15,000 ratings, from AusDM2009 competition, and can be found @ Here .

Using example code from the book, it takes 1563 seconds on a regular Windows desktop (Core2Duo E6750 2.67GHz, 4GB Memory, 7K2 rpm HDD with 8MB cache) to process 5 numerical continous variables, whereas with improved macro, it only takes 1 second to process the same amount of data. This improvement is criticle since weak classifiers like this one won't be used alone, but as the base for more time-consuming Boosting algorithms. With original macro, it is practically not usable for any boosting algorithms on data sets with hundreds of or more observations.

Original Macro:

Improved Macro:

Comparing the results from original macro and the improved macro, they both select X3 with the same partition cut off point and the same Gini Index.

Reference:
Pharmaceutical Statistics Using SAS: A Practical Guide by Alex Dmitrienko, Christy Chuang-Stein, Ralph B. D'Agostino, SAS Publishing 2007

SAS Programming: SAS implementation of Kernel PCA

Tue, 16 Mar 2010 01:32:45 +0000

Kernel method is a very useful technique in data mining that is applicable to any algorithms relying on inner product [1]. The key is applying appropriate kernel function to the inner product of original data space.

I show here SAS/STAT+BASE example code for Kernel PCA implementation. The example used is from Wikipedia @ here. Extention to other algorithms that suitable for Kernel method, such as CCA, ICA, LDA, etc, is straightforward.

Reference:
[1]. Zhu, Mu, "Kernels and Ensembles: Perspectives on Statistical Learning", The American Statistician. May 1, 2008, 62(2): 97-109

Figure 1. Linear PCA Result.

Figure 2. Result of Kernel PCA of Inner Product on quadratic polynomial formula

Figure 3. Result of Kernel PCA of Frobenius Norm on Gassian Kernel

SAS demo code:


/* Demonstrate kernel PCA. Kernel method can be applied to any algorithm that relies on inner product */
data original;
     do ID=-314 to 314;
        x1=sin(ID/100)*1+rannor(8976565)*0.1; x2=cos(ID/100)*1+rannor(92654782)*0.1; Class=1; output;
        x1=sin(ID/100)*2+rannor(8976565)*0.1; x2=cos(ID/100)*2+rannor(92654782)*0.1; Class=2; output;
        x1=sin(ID/100)*3+rannor(8976565)*0.1; x2=cos(ID/100)*3+rannor(92654782)*0.1; Class=3; output;
     end;
run;

/*------------ Linear PCA -----------*/
proc princomp data=original  standard noint cov  
              outstat=lin_stat(where=(_TYPE_ = 'USCORE'))
              noprint;
        var x1 x2;
run;


data lin_stat_score;
     set lin_stat;
     _TYPE_= 'PARMS';
run;

proc score data=original   score=lin_stat_score  type=parms   out=lin_pca;
     var x1 x2;
run;

proc gplot data=lin_pca;
        plot Prin1*Prin2=Class;
run;quit;


/*--------- Kernel PCA 1.0 ------------*/
/* Inner product based kernel PCA */
proc transpose data=original(drop=ID  Class) out=original_t; run;

proc corr data=original_t  outp=inner(where=(_TYPE_='SSCP' & substr(_NAME_, 1, 3)='COL')  drop=intercept)  sscp noprint;
     var col:;
run;

/* apply kernel functon to inner product matrix. Here we use (X'Y+1)^2, the one used in wikipedia.org example. */
data inner;
     set inner;
     array _C{*} _numeric_;
     _TYPE_='COV';
     do j=1 to dim(_C); _C[j]=(_C[j]+1)**2; end;
     drop j;
run;

proc princomp data=inner    noint  cov  standard 
              outstat=k_stat(where=(_TYPE_ in ('EIGENVAL', 'USCORE')) )
              noprint;
     var col:;
run;

data S(drop=_NAME_)  score;
     set k_stat;
     if _TYPE_='EIGENVAL' then output S; else output score;
     drop _TYPE_;
run;

data _null_;
     set S;
     array _S{*} _numeric_;
     do j=1 to dim(_S);
        if _S[j]<0.5*constant('MACEPS') then do;
          call symput('maxsingval', j-1); stop;
        end;
     end;
run;
%put &maxsingval;

data score; set score; if _n_>&maxsingval then delete; retain _TYPE_ 'PARMS';  run;

proc score data=inner   score=score  type=parms  out=k_U(keep=Prin:);
        var col:;
run;

data k_U;
     set original(keep=ID Class  x1 x2)  ;
     set k_U;
run;

proc gplot data=k_U; 
     plot Prin1*Prin2=Class;
run;quit;


/*--------- Kernel PCA 2.0------------*/
/* Frobenius norm based kernel PCA */
proc transpose data=original(drop=ID  Class) out=original_t; run;

proc corr data=original_t  outp=inner(where=(_TYPE_='SSCP' & substr(_NAME_, 1, 3)='COL')  drop=intercept)  sscp noprint;
     var col:;
run;

proc means data=original_t noprint ;
         var col:;
         output out=_uss(drop=_TYPE_    _FREQ_)    uss= /autoname;
run;

proc transpose data=_uss  out=_usst; run;

proc sql noprint;
          select nobs into :NCOLS from sashelp.vtable
          where libname='WORK'
              and memtype='DATA'
              and memname='_USST'
              ;
quit;
%let ncols=%sysfunc(compress(&NCOLS));
%put &ncols;

/* apply kernel functon to inner product matrix. Here we use Gaussian kernel function. */
data inner;
          array _X{&ncols} COL1-COL&ncols;
          array _USS0{&ncols}  _temporary_;
          if _n_=1 then do;
             do j=1 to &ncols;
                  set  _usst(keep=COL1)  point=j;
                  _USS0[j]=COL1;
             end;
          end;
          set inner;
          _TYPE_='COV';
          do j=1 to &ncols;
              _X[j]=_USS0[_n_] + _USS0[j] - 2*_X[j];
              _X[j]=exp(-0.5*_X[j]/&sigma);
          end;
          keep _TYPE_  _NAME_ COL1-COL&ncols;
run;


proc princomp data=inner    noint  cov  standard 
              outstat=k_stat(where=(_TYPE_ in ('EIGENVAL', 'USCORE')) )
              noprint;
     var col:;
run;

data S(drop=_NAME_)  score;
     set k_stat;
     if _TYPE_='EIGENVAL' then output S; else output score;
     drop _TYPE_;
run;

data _null_;
     set S;
     array _S{*} _numeric_;
     do j=1 to dim(_S);
        if _S[j]<0.5*constant('MACEPS') then do;
          call symput('maxsingval', j-1); stop;
        end;
     end;
run;
%put &maxsingval;

data score; set score; if _n_>&maxsingval then delete; retain _TYPE_ 'PARMS';  run;

proc score data=inner   score=score  type=parms  out=k_U(keep=Prin:);
        var col:;
run;

data k_U;
     set original(keep=ID Class  x1 x2)  ;
     set k_U;
run;

proc gplot data=k_U; 
     plot Prin1*Prin2=Class;
run;quit;

Further Reading:
Chapter 1 & 2 of
Principal Manifolds for Data Visualization and Dimension Reduction by A. Gorban, B. Kegl, D. Wunsch, A. Zinovyev (Eds.) Lecture Notes in Computational Science and Engineering, Springer 2007

SAS Programming: Transform a Hierarchical Long To Wide Format with Large Number of Heterogeneous Variables

Tue, 16 Mar 2010 01:27:56 +0000

SAS Programming: Parallel Processing with Single-Thread SAS license

Tue, 16 Mar 2010 01:12:02 +0000

[NESUG 22]

SAS now support Multi-threads Multi-processor in various PROCs. To fully leverage multi-processor speedup, a server version SAS with Multi-Thread support, or SAS/MP Connect modular is necessary which is very expensive. Is there a way to get around with only SAS/Base/STAT/ETS?

While not fully capable to utilize multi-CPUs multi threads in all jobs, the method introduced here are good for some computing-intensive tasks that are popular in modeling and anlysis, such as Bootstrapping, Cross-Validation and Simulation. These methods involve almost identical computation on multiple independent replicate samples/subsamples, thus, we can decompose them into multiple independent computation pieces.

We present two simple bootstrapping demos here, using the data and program from SAS Documentation for PROC NLMIXED example 51.1.

The first method used "x" command to execute a Batch job in DOS.

The main program will call batch jobs using "x command/statement" with concurrent mode where mutliple SAS jobs can access the same sas data set. In order to differentiate different pieces of the computing job, SAS automatic variable -SYSPARM is introduced to pass relavent information. Of course, the operating data sets must be stored in one or multiple permanent libraries so that it is accessible for all batch jobs.

The macro will be executed in batch mode. Macro variables used to differentiate different jobs are passed from system automatic variable -SYSPARM. SYSJOBID can also be used for the same purpose. SAS Institute Inc. provide a comprehensive tutorial[2] on how to set up Batch Jobs and execute them in SAS, and Jackson explained how to use -SYSPARM in Batch mode [1]. Kosian, Sassoon called this method "virtual multi-threading" [5].

This method, however, has a major issue, that is it is hard to control and check the status of the batch jobs. Multiple Stage jobs could easily get coding a very ugly work.

Suppose we are going to do further analysis based on part or all of previous jobs's results, and the execution time of these jobs various, then, the main SAS program needs to wait for all these needed jobs to complete in order to continue. This Batch Execution using "X" command doesn't provide an easy way for current SAS session to check wether these batch jobs have been finished correctly. Note that we can't wait for them to execute sequentially. One possible method is to ask the batch jobs to write to a txt file when it is finished, and a follow up macro program in the current SAS session will use a NULL data step to check this txt file every, say, 2 minutes. This can be done via a sleep function in the NULL data step. Once all needed job finishing codes were read from this txt file, following SAS jobs can be executed.

Obviously this is very inconvenient. Fortunately, SAS provides a unified production framework to help us complish this task [3].

In method 2, we utilize the 'systask command' statement to execute SAS jobs. The syntax is:

SYSTASK COMMAND "operating system-command"
WAIT | NOWAIT
TASKNAME=taskname
MNAME=name-var
STATUS=stat-var
SHELL<="shell-command">;

Consult with SAS Documents for detailed information.

The most relavant options are TASKNAME and STATUS. TASKNAME specifies a name that identifies the task, which then is used with WAITFOR statement to pause the current SAS session until the tasks finished. STATUS specifies a macro variable in which you want SYSTASK to store the status of the task. A sample demo is as following:

Note that this method is able to complete most of the Multi-Processor jobs that SAS/MP Connect was specifically built for, which also requires parallelized jobs be independent. Bentley [4] showed several typical production jobs that can be boosted using parallel computing, but resort to expensive SAS/MP Connect software.

In conclusion, we showed that with a bit more coding and careful design of the job flows, we are able to utilize multi-core capability of moder computer without resort to more expensive SAS modular.

For comparison, check the CPU usage statistics when using and not using the method proposed here.
=> Single Thread PROC NLMIXED;

=> Make It Two Threads

--Ref:
1. Jackson, Clarence Wm., "Using SAS® SYSPARM And Other Automatic System Variables", NOTSUG May 2009;
2. SAS Technical Notes TS-648, "Examples of Batch Processing under Windows", SAS Institute Inc.
3. Cogswell, Denis, "More Than Batch – A Production SAS® Framework", SUGI 30, Applications Development
4. Bentley, John E., "SAS Multi-Process Connect: What, When, Where, How, and Why", SUGI 26, Systems Architecture
5. Kosian, Sassoon, "A Generic Method of Parallel Processing in Base SAS® 8 and 9", SUGI 27, Coder's Coner

====================================================
FYI:
Federal Government contract SAS license and renewal fee comparison between 1 PC user and multi users [Public Information]. Now, you save something nontrivial.

SAS Programming: Randomly Split SAS table exactly according to a given probability vector

Tue, 16 Mar 2010 01:02:56 +0000

*K/N Algorithm based spliting given probability vector

*Ensure randomness within any level of value combination of control variables

*Only need to pass through data once in spliting step


/*************************************************/
%macro split(
                     seed,           /*seed for random number generator                       */
                     probvector,   /*macro variable of spliting prob vector separated by "  " */
                     control_vars, /*control variable as in PROC SURVEYSELECT                 */
                     in_dsn,         /*input data set                                         */
                     out_dsn=_out_dsn_, 
                     sort=nest
                     );
%let blank=%str( ); 
%let n=1; %let nv=1;
%let probs=&probvector;

%let dsid=%sysfunc(open(&in_dsn));
%if  (&dsid=0) %then %do;
     %put %sysfunc(sysmsg());
     %put Stop Processing due to Errors;
     %goto exit;
%end;
%else %do;
    %let nobs=%sysfunc(attrn(&dsid, NLOBS));
    %let dsid=%sysfunc(close(&dsid));    
%end;

%if &out_dsn=&blank %then %let &out_dsn=&in_dsn;
%if &seed=&blank %then %let seed=0;

%let pos=%scan(&probs,&n,&blank); 
%do %while(&pos ne &blank); 
    %let n=%eval(&n+1); 
    %let pos=%scan(&probs,&n,&blank); 
%end;
%let n=%eval(&n-1);

%let pos=%scan(&control_vars,&nv,&blank);
%let control_vars_x=&pos;
%let control_key=%str(&pos);
%let control_find=%str(key:&pos); 
%do %while(&pos ne &blank); 
    %put &pos;
    %let nv=%eval(&nv+1); 
    %let pos=%scan(&control_vars,&nv,&blank); 
    %if &pos^=&blank %then %do;
        %let control_vars_x=%str(&control_vars_x * &pos);
        %let control_key=%str(&pos%", %"&control_key);
        %let control_find=%str(key:&pos, &control_find);    
    %end;
%end;
%let nv=%eval(&nv-1);
%put nv=&nv;
%if %eval(&nv>0) %then %do;
    %if %upcase(&sort)=NEST %then %do;
        proc sort data=&in_dsn out=_in_dsn_; 
             by &control_vars; 
        run;
    %end;
    %else %if %upcase(&sort)=SERP %then %do;
          ods select none;
          proc surveyselect data=&in_dsn samprate=1.0 out=_in_dsn_
                            sort=&sort  method=sys;
               control  &control_vars;
          run;
          ods select all;
    %end;
    %else %do;
         %put Specify only NEST or SERP for sort statement;
         %goto exit;
    %end;
    %let in_dsn=_in_dsn_;

    proc freq data=_in_dsn_ noprint;
         tables &control_vars_x /missing out=_freq_;
    run;

    proc sql noprint; 
           select name into :keyvars separated by '  '    
           from  sashelp.vcolumn
           where libname='WORK' & 
                     upcase(memname)='_FREQ_' & 
                     name not in ('COUNT', 'PERCENT')
           ;
    quit;
    data  _freq_(index=(idx=(&keyvars)));
            set _freq_  nobs=ntotal  end=eof;
            _index_=_n_;
            if eof then call symput('ncomb', ntotal);
    run;
    %let withcontrol=1;

%end;
%else %do;
    %let withcontrol=0;
    %let ncomb=1;
%end;

proc sql noprint;
       select name into :originalvars separated by ' '
       from  sashelp.vcolumn
       where libname='WORK' & 
                     upcase(memname)='_IN_DSN_'
      ;
quit;

data probs;
     %do i=1 %to &n;
         _prob_&i=%scan(&probs, &i, &blank);
     %end;        
     output;
     drop i;
run; 

%let i=&n;
%put Initial counts: n=&n  i=&i   ncomb=&ncomb;
data &out_dsn      _N;
     array _P{*}  _prob_1-_prob_&n;
     array _N{0:%eval(&n+1)}  N_0-N_%eval(&n+1);
     array _Pmat{1:&ncomb, 1:&n} _temporary_;     
     array _Fmat{1:&ncomb, 1:&n} _temporary_;
     array _Xmat{1:&ncomb, 1:&n} _temporary_;
     array _Kmat{1:&ncomb, 1:&n} _temporary_;
     array _Nmat{1:&ncomb, 0:%eval(&n+1)} _temporary_;    

     if _n_=1 then do;
        set probs;        
        do i=1 to &ncomb;
           do j=1 to &n;
              _Pmat[i, j]=_P[j];
              if j=1 then _Fmat[i, j]=_P[j];
              else _Fmat[i, j]=_Fmat[i, j-1]+_P[j];
           end;
           do j=1 to &n;
              _Fmat[i, j]=_Fmat[i, j]/_Fmat[i, &n];
           end;
        end;
        put 'FINISH LOADING PROB VECTORS';
        %if &withcontrol=1 %then %do;            
            do while (^eof);
               set  _freq_  end=eof;              
               cn=0; 
               _Nmat[_index_, 0]=COUNT; _Nmat[_index_, %eval(&n+1)]=0;   
               do j=1 to &n;
                  if j<&n then do;
                     _Nmat[_index_, j]=round(_P[j]*COUNT, 1);
                     cn+_Nmat[_index_, j];
                  end;
                  else _Nmat[_index_, j]=COUNT-cn;
                  _Xmat[_index_, j]=_Nmat[_index_, j];
                  _Kmat[_index_, j]=0;
               end;               
            end;
        %end;    
        %else %do;
            _index_=1; cn=0;
            do j=1 to &n;
               if j<&n then do;
                  _Nmat[_index_, j]=round(_P[j]*&nobs, 1);
                  cn+_Nmat[_index_, j];
               end;
               else _Nmat[_index_, j]=&nobs-cn;
               _Xmat[_index_, j]=_Nmat[_index_, j];
               _Kmat[_index_, j]=0;
            end;
        %end;
        put 'FINISH LOADING ALL DATA SETS';
        do i=1 to &ncomb;
            do k=0 to %eval(&n+1);               
                _N[k]=_Nmat[i, k]; put _Nmat[i, k]=;
            end;
            output _N;
        end;
     end;
     set &in_dsn  nobs=ntotal   end=eof_in;

     r=ranuni(&seed);
     %if &withcontrol=1 %then %do;
        do until (_iorc_=%sysrc(_dsenom));
            set _freq_  key=idx;
            if _iorc_^=%sysrc(_sok) then  _error_=0;        
        end;        
     %end;
     %else %do;
         _index_=1;
     %end;
     notfound=1;  j=1;

     do j=1 to &n;
         if notfound then do;
               if r<=_Fmat[_index_, j] & _Kmat[_index_, j]<_Nmat[_index_,j] then do;
                    _BLOCK=j; notfound=0; _Kmat[_index_, _BLOCK]+1; 
               end;
        end;        
     end;
     if notfound then do;  _BLOCK=&n; notfound=0; _Kmat[_index_, _BLOCK]+1; end;

     _Xmat[_index_, _BLOCK]=_Nmat[_index_, _BLOCK]-_Kmat[_index_, _BLOCK];
     _Nmat[_index_, %eval(&n+1)]+1; 

     if ^(eof_in ) then do;
        _temp_=(_Nmat[_index_, 0]-_Nmat[_index_, %eval(&n+1)]);
        if _temp_<=0 then _Pmat[_index_, _BLOCK]=0;
        else  _Pmat[_index_, _BLOCK]=_Xmat[_index_, _BLOCK]/_temp_;               
        do j=1 to &n;
           if j=1 then _Fmat[_index_, j]=_Pmat[_index_, j];
           else _Fmat[_index_, j]=_Fmat[_index_, j-1]+_Pmat[_index_, j];  
        end;
        do j=1 to &n;   
            if _Fmat[_index_, &n]<=0 then _Fmat[_index_, j]=0;
            else _Fmat[_index_, j]=_Fmat[_index_, j]/_Fmat[_index_, &n]; 
        end;
     end;
     else if (eof_in) then do;
        do i=1 to &ncomb;
            do k=1 to %eval(&n);               
                _N[k]=_Kmat[i, k]; put _Kmat[i, k]=;
            end;
            output _N;
        end;
     end;
     keep &originalvars  _BLOCK;
     output &out_dsn;
run;

%exit:
%mend;


*options mprint mlogic;
%put START TIME 1: %sysfunc(datetime(),datetime20.);

data test0;
     do ID=1 to 796773;        
        TDSP=min(3, round(ranuni(8976)*4));
        if ranuni(93745)<0.3 then VS='H'; else VS='M';
        output;
     end;
run;
%put START TIME 2: %sysfunc(datetime(),datetime20.);

%let seed=99999;
%let probvector=0.1485905 0.1485905 0.2324283 0.2351954 0.2351954;
%let control_vars=TDSP  VS;
%let in_dsn=test0;
%let out_dsn=test_out;
%let sort=NEST;
%split(&seed, &probvector, &control_vars, &in_dsn, out_dsn=&out_dsn);
*options nomprint nomlogic;
%put END TIME: %sysfunc(datetime(),datetime20.);

;;

proc freq data=&out_dsn noprint;
        tables _BLOCK*TDSP*VS /missing out=_freq_out_;
run;

SAS Programming: SVD Filtered Temporal Usage Pattern Analysis and Clustering

Tue, 16 Mar 2010 01:01:58 +0000

Jared Prins' Blog: Impressed with SAS Text Miner Help Files

Mon, 15 Mar 2010 15:58:27 +0000

As a tech junkie, people in the office often come by to ask for help with various software. As this is not a part of my job description and I have deadlines to meet, sometimes my response is "Press F1". Of course, this will usually open the help file for that software.

Two weeks ago I upgraded SAS Enterprise Miner to 6.1 and SAS Text Miner to 4.2. Although painful to upgrade, it was worth it. The new TM has all new TM nodes and both products seem much more polished.

After spending all this time getting familiar with the old TM, I now needed to learn new TM functionality. I began to look for documentation since all of my books and training manuals were now partially out of date. I turned to the web which had virtually no helpful TM 4.2 documentation, I'm guessing because TM 4.2 is too new? (Or perhaps yours truly was googling with poor search terms).

Not sure where to turn, I walked over to the boss's office. That's when I got a taste of my own medicine. "Press F1", he said.

Of all the help files I've ever seen, SAS help files are the most impressive. There's about a hundred pages of in-depth descriptions for the new nodes plus examples of the TM process. Whoever at SAS writes these help files - Thank you, thank you, thank you!

The SAS Dummy: Multiple identities: for all of the people you are

Mon, 15 Mar 2010 13:18:41 +0000

When you connect to a SAS environment with SAS Enterprise Guide, are you a single, consistent person? Or do you have more identities than Chevy Chase in Fletch?

When you connect to a SAS environment, you can be many people at once:

You are logged into your Windows account on your PC
You connect to SAS metadata using a metadata identity
You connect to a SAS workspace session with a host identity on the workspace server.

If you have a single sign-on configuration, then your identity should be the same across the board. But if not, the accounts used in the various tiers might be different. Here is a simple SAS program that can help you to answer the deep philosophical question: Who am I?



/** for use on Windows and Unix workspace servers **/

%let _metauser = %scan(%sysget(METAUSER),1,'@');

/** for use on MVS, Stored Process, and Pooled Workspace servers **/

/** %let _metauser = %scan(%sysfunc(getoption(METAUSER)),1,'@'); **/

data ids;

  length name $ 16 purpose $ 40;

  label name="Value Name" purpose="Purpose";

  infile datalines dsd;

  input name purpose;

  datalines;

SYSUSERID, SAS session host account

_CLIENTUSERID, Windows user ID in SAS EG

_CLIENTUSERNAME, Windows user name in SAS EG

_METAUSER, SAS metadata user ID

run;



proc sql;

  select t1.name, 

    t2.value, 

    t1.purpose

    from work.ids t1 

    inner join sashelp.vmacro t2 on (t1.name = t2.name);

quit;

When you run this program in SAS Enterprise Guide, you'll produce a simple report that shows all of the IDs that make you the special "you" that you are to SAS.

The Princess of Science: ‫יום פיי שמח!‬

Sun, 14 Mar 2010 13:05:35 +0000

‫המתמטיקאים בעולם חוגגים היום את יום הפיי. פיי הוא קבוע מתמטי שודאי שמעתם עליו, וערכו שווה בקירוב ל-3.14.בשיטה הנהוגה בארה"ב, התאריך של היום, ה-14 במרץ, נכתב כך: 3.14, ומכאן מקור המנהג. בגוגל מציינים את היום על ידי לוגו מיוחד (שהוא התירוץ לכל הפוסט הזה): ואם כבר כתבתי פוסט, אז הנה חידון פיי שכתבתי פעם כאשר ניהלתי את [...]‬

StudySAS Blog: PRXMATCH Function | StudySAS Blog [Digg]

Sat, 13 Mar 2010 03:14:56 +0000

The SAS Dummy: "SAS Talks" is home

Fri, 12 Mar 2010 15:24:17 +0000

The SAS Talks series is a big hit, and now we've got a landing page where you can find previously recorded talks, plus information about upcoming talks.

You can learn about SAS programming with SAS Enterprise Guide, ODS statistical graphics, fancy multi-sheet output for Microsoft Excel from SAS, and more!

They're free and available on-demand!

JMP Blog: A Hot Time at Pittcon

Thu, 11 Mar 2010 23:00:10 +0000

Until last week, it had been a while since I've been to a trade show.

My colleague and favorite chemist, Lou Valente, asked me to join him in the JMP booth at Pittcon, the Pittsburgh Conference.

The Pittsburgh Conference is a huge event with 20,000+ chemists and other laboratory scientists sharing ideas and best practices. The expo floor is filled with lab instrument and equipment manufacturers and others, like us -- software developers. This year, the Pittsburgh Conference was in Orlando, FL. (I guess would-be scientists don't have to take much geography.)

Our booth was directly across the aisle from Control Company. They design, manufacture and sell traceable laboratory equipment and scientific instruments -- things like thermometers, anemometers, calipers and even clocks.

I didn't know what "traceable" meant in this context, but they provide a certificate that can trace the calibration of instruments back to National Institute of Standards and Technology. Pretty cool.

Here's a picture of one part of their booth. Notice the lights across the top. They had a 10x30 foot booth with about 40 of the halogen lights you see at the top of the booth panels. That meant that their booth, and because of the proximity, our booth, was pretty warm.

Check out this close-up of some thermometers they had on display. Those temperatures are in Celsius. To save you the trouble of converting it for yourself, I'll point out that the 37 degrees Celsius shown on the top row of thermometers is 98.6 degrees Fahrenheit. And, while that's a normal temperature inside the human body, it's a little warm for a trade show floor. On the first day of the show, we saw that top row of thermometers reach 40 degrees (104 Fahrenheit!).

So, on the second day of the show, we started logging the temperatures of the top row and bottom row of the thermometers shown in the picture above. Here are those three days in Graph Builder in JMP.

You can see that on Days 3 and 4 of the show, we started collecting data before they turned the lights on. They heat up pretty quick. On Day 4, the show ended at 2:00 p.m., so we didn't get any data after that.

I'm not sure why the curve for Day 2 of the show is different from the other two days. They probably had the air conditioning on that day and not the other two since it was cooler outside those days. Regardless, we had fun using this to show how quickly we could take the data and create an interesting look at it.

the SAS-BI blog: Developing Custom OLAP Cube Aggregations

Thu, 11 Mar 2010 09:00:08 +0000

Business problem: By default, OLAP cubes calculate aggregations at each possible intersection of levels in one or more dimensions. However, there are situations where aggregations cannot be completed on the fly for all intersections within the cube due to business rules for some data levels. Also, the source data table isn't a fully summarized table (which fits nicely into the OLAP Studio GUIs) and derived measures are not possible due to specific business rules.

Solution: Within the PROC OLAP code, you can specifically assign an individual table for each Aggregation. This option is defined in the online help at:
http://support.sas.com/documentation/cdl/en/olapug/59574/HTML/default/a002605627.htm
It is important to note that the individual aggregated tables must have columns that completely match those listed in the aggregation step & this table must have at least one column defined in the MEASURE AGGR_COLUMN= statement.
Contributed by:
J. Alan Stark of Ashlind Consulting
http://www.ashlind.com/

JMP Blog: Visual Data Quality with Named Colors in JMP, Part 2

Wed, 10 Mar 2010 18:29:00 +0000

After my previous exploration of colors and names revealed inconsistencies in the Wikipedia color data, I looked around for a more authoritative source of color names. No luck finding an oracle, but I did find another interesting data set.

Where the Wikipedia table provided color values for a given set of names, this data set provides names for given color values. Dolores Labs created a Mechanical Turk task to have people give names to the color swatches they were assigned to. The results are provided in a CSV file, which JMP opens without any trouble at all.

As you might expect, there are lots of duplicate names. Out of about 10,000 color values, 1,000 were assigned the name "blue." Conversely, sometimes the same color value would be assigned different names, such as "purple" and "violet." And since the names were entered by the workers, instead of picked from a list, there is some variation in spelling. Column Recode is really handy for fixing up some of these problems. Here is a subset of the names after applying Trim Whitespace and Title Case in Col : Recode.

Color names before and after

I also used Find and Replace to change "Drk" to "Dark" and other abbreviations. That still left about 1,500 unique names, most of them occurring once. I wanted to see how much variation there was in colors of the same name, so I first filtered out the names with fewer than 15 color values. To do that, I used Summarize on the color name column, which told me how many colors had each name. Then I could sort the table by the counts and select the rows with fewer than 15 colors. Since the tables are linked, that also selects those in the original table. After excluding those, I plotted the remaining in Graph Builder as Hue by Name.

The graph is long, so I'll leave it to the end, but if you look closely (click the graph to see it full size), you might notice things like a purple called green and a green called blue. The names are ordered by average hue, which sometimes doesn't work too well for the reds, which often legitimately straddle the edge between 0° and 360°. So this is another example where a visualization helps detect data quality issues.

StudySAS Blog: $UPCASEw. format

Wed, 10 Mar 2010 17:37:36 +0000

We all know the importance of UPCASE function in handling the character case strings. But do you know that a format can let you do the same as the UPCASE function (upcasing the variables).$UPCASEw. format works similar to the UPCASE Function. It also does one more thing which UPCASE function doesn’t. i.e: $UPCASEw. format let you apply length to the variable. Remember that w specifies the width of the output field. Example:*********************************************************; data new;*convert it to uppercase;name="studysas blog"; format name $upcase.;newname=put(name, $upcase32.);*Put function let you apply $upcase format;run;**********************************************; *The length of the new variable newname will be 32.

[[ This is a content summary only. Visit my website for full links, other content, and more! ]]

JMP Blog: Changing JMP Graph and Selected Text Colors (for Windows OS)

Tue, 09 Mar 2010 14:00:00 +0000

At Sam Gardner’s March 4 Webcast on JMP Journals, we closed with a few basic unanswered questions about changing color settings in JMP for Windows. I promised the answers.

1. You can change most graph settings globally from JMP Preferences. For example, to change most graph attributes, including background color, marker size, marker style and color schemes, choose PREFERENCES>REPORTS, then make and apply your changes.

2. To change the background color of tables, choose PREFERENCES> WINDOWS SPECIFIC>WINDOWS BACKGROUND COLOR.

3. To change the color used to highlight selected menu options or table rows that correspond to areas of a graph that you highlight, you need to change a Windows setting. NOTE: Because these settings will apply to all Windows programs, you may want to note the defaults before changing.

From your Windows Settings Control Panel, select DISPLAY>APPEARANCE>ADVANCED. From the ITEMS box, chooses SELECTED ITEMS, and then choose your color. You may also need to change the FONT color. For example, yellow highlight was unreadable with my default white font, so I changed FONT color to black.

Looking for other tips? Try our searchable JMP Knowledge Base & FAQs.

the SAS-BI blog: Alert Notification from BI Dashboard 4.2? Include EMAIL Addresses in User Manager

Mon, 08 Mar 2010 09:00:02 +0000

If you would like for the BI Dashboard to send you an alert via email, you must ensure that the email address is included for each user that will be receiving emails. This is located in SAS Management Console User Manager.

Instructions: http://support.sas.com/documentation/cdl/en/biwaag/63059/HTML/default/a003311779.htm
Example:

The SAS Dummy: Just testing!

Fri, 05 Mar 2010 15:57:43 +0000

Yesterday, we at SAS performed a little experiment, unintentionally. Who would notice (we wondered in retrospect) if we disabled one of our discussion forums on support.sas.com?

As we learned: a lot of people would notice.

There was a technical glitch that broke the SAS Enterprise Guide discussion forum for much of the day yesterday. All of the other discussion forums (and there are a bunch) were unaffected. But it warmed my heart to hear that so many people missed discussing SAS Enterprise Guide topics with their fellow users.

Thanks for participating in the forums. And thanks for letting us know that you missed us during the brief outage.

JMP Blog: Quick Reference Card

Thu, 04 Mar 2010 20:28:36 +0000

Do you love keyboard shortcuts as much as I do? If so, be sure to check out the Quick Reference Card (found in JMP by going to the Help menu and selecting Books -> Quick Reference Card). It is a PDF of the keyboard shortcuts available in JMP, broken down by the operating system (Windows, Macintosh, or Linux). Many of them will be familiar to you because they are common keyboard shortcuts for the applicable operating system, but a number of them are unique to JMP. For example, did you know you can increase the font size by holding down the CTRL and SHIFT keys and then pressing the plus sign? Similarly, pressing the minus sign will decrease your font size. Also, did you know you can hold down the ALT key and press the first letter of a menu title to open that menu? The only exception is the Tools menu, which requires ALT+O. As another example, you can also move your current window to the back with CTRL+B. Do you need to create a new data table? CTRL+N. Do you need to show the log window? CTRL+SHIFT+L. Do you want to add the current report to a journal? CTRL+J. The list goes on and on…

In addition to keyboard shortcuts, the Quick Reference Card also features common clicks and special functions. Examples of some common clicks include holding the CTRL key down while resizing a graph so that you can resize all similar graphs the same way and right-clicking on a selected cell in a data table and selecting Fill to repeat values or a sequence of values in a column. Examples of special functions include how to check that fences match in a script (place the cursor to the left of an opening fence or to the right of a closing fence and both fences will highlight if a match is found) and how to select multiple analysis options from a red triangle menu (hold the ALT key down before clicking the red triangle).

Finally, you can find a detailed description of the options in the Formula Editor and the Tools menu. These components are especially helpful if you are a new JMP user who is learning to navigate the interface and learning what the different options mean. You might like to print a copy of the Quick Reference Card and have it handy as you learn more about JMP.

JMP Blog: See You at SAS Global Forum

Thu, 04 Mar 2010 16:17:29 +0000

It’s like I won a golden ticket to Willy Wonka’s chocolate factory. That’s what it feels like to be a SAS employee who’s asked to go to SAS Global Forum, April 11-14 in Seattle. The buzz on the demo-room floor, hearing from thought leaders both entertaining and enlightening, the interaction between the brilliant developers and the genius folks who use the software … it’s really an action-packed few days.

But with that golden ticket comes a lot of responsibility. In addition to providing on-site coverage for this blog, I’ll be reconnecting with customers who have been using JMP for years as well as hosting members of the press who are just learning about JMP. But my self-imposed goal is to leave SAS Global Forum with no fewer than 10 new acquaintances, at least five of whom are using JMP and SAS together. I want to learn about their applications, and I want to turn those applications into success stories for the rest of the world to see.

So, if you’re using JMP and some other SAS product together, I want to meet you.

After you take in Stephen Few’s post-conference seminar and after you’ve been dazzled by my boss, Jon Weisz, at the Technology Connection, come find me. Seek me out between the papers presented by my JMP friends and colleagues, including my big boss, John Sall. When you build your agenda, build in time to share your story with me.

You’ll find me on the demo-room floor or wherever JMP papers are being presented. I look forward to seeing you in Seattle!

SAS Global Forum: SAS CEO Jim Goodnight Extends Special Invitation

Wed, 03 Mar 2010 21:23:00 +0000

As Dr. Goodnight explains, SAS Global Forum (formerly known as SUGI) was started by SAS users, who held their first conference in 1976, several months before SAS was even incorporated as a company. The successful tradition of a user run conference, guided by an Executive Board of SAS users, continues today, 35 years later.

If you haven’t registered already, be sure to do so soon. Early registration ends March 8! Register now, and you can save up to $250 off your registration fee. There are additional ways to save as well. Fees will increase on March 9 for all Registration Packages.

This year will be my 14th conference, and I totally agree with how Dr. Goodnight sums things up in his invite video. “[SAS Global Forum is a...] huge collaborative forum where problems are solved, techniques are shared, and life-long friendships are made.”

SAS Global Forum: Michael Raithel's plans for SAS Global Forum

Wed, 03 Mar 2010 20:57:00 +0000

While at SAS, I hope to get the opportunity to talk with all of the SAS icons. I’ve had the privilege to meet many. Before the SAS winter break, I had the privilege to talk with another: Michael Raithel.

At SAS® Global Forum 2010, Raithel will be presenting the Tuesday lunch feature presentation, “It’s not easy being a SAS programmer." So, I thought you might like to know a little more about who he is and what he does.

Who he is
Raithel is a SAS programmer, former NESUG conference chair, former SUGI section chair, three-time SAS author and a Senior Systems Analyst at Westat. Senior Systems Analyst … how many Senior Systems Analysts have you met? When I was at NESUG (my first regional), I met many. Raithel’s answer to my quip said it all, though. “If they called me ‘Senior Dog Catcher,’ it wouldn’t matter. I have a wonderful, fulfilling job here. This is an excellent place to work. There is relatively low turnover and very high job satisfaction; a very professional and serious, yet respectful, work environment.”

So far, he seems pretty much rank and file with the other SAS icons, doesn’t he? Raithel also has at least one other cool point: His first book, Tuning SAS Applications in the MVS Environment, is now part of the Smithsonian Institution’s Museum of American History Information Technology Collection. Well now, that’s the stuff of legends. Wouldn’t you agree?

SAS crunchers and munchers
Westat is an employee-owned corporation that provides contract research services to businesses, foundations, US government agencies, and state and local governments. A typical example of work Westat does is to host, manage and clean data for a state cancer registry. “SAS is an integral part of the software we use to manage the data” said Raithel. “Most of the other hundreds of research projects conducted for clients each year by Westat use SAS products.”

“We’re basically a meat and potatoes kind of SAS shop,” said Raithel. “By that, I mean we have 18 SAS products that are the foundation of much of our work. We have deployed four SAS/ACCESS® interfaces, SAS/CONNECT®, SAS/GRAPH®, SAS/Genetics™, SAS/IML®, SAS/Intrnet, and SAS/STAT®, to name a few. We are also looking to expand our use of SAS to include SAS® Data Quality.”

Westat may consider itself a “meat and potatoes SAS shop,” but the company is proving that treating people well leads to success. Success is promoted and encouraged by a support infrastructure that includes an in-house technical support unit and Westat-written SAS resources.

“On our intranet, we have SAS Resources Web pages that give our users one-stop shopping to learn what SAS is all about at Westat, including links to contact information for the people in my group, Westat’s SAS technical support,” said Raithel. “The pages include instructions for loading SAS on a desktop, SAS resources and Westat conference papers, documentation for the SAS products we use, the methods we use for validating hot fixes and even the date and location of the next Westat SAS Users Group meeting.”

According to Raithel, Westat’s SAS technical support department, which he heads, helps to ensure that SAS Institute's Technical Support doesn’t receive a flood of calls each day from Westat. Raithel and his two staff members “answer everything from, ‘I can’t open SAS today,’ to ‘What is happening with SAS 9.2?’ and ‘This report looks kind of funny.’”

When Raithel and his staff have questions they can’t answer, they usher them through the official SAS Technical Support process. “When there is a satisfactory result, and we believe that question is going to affect more users, we post it to the in-house listserv – SAS Outlook Information Forum,” he said. “SAS users at Westat also post questions and answers on the listserv.”

Success isn’t guaranteed, it’s mastered
Each year, Westat purchases SAS training EPTO units. “We buy enough EPTO units per year to allow us to have custom-tailored classes taught in our facilities,” said Raithel. “SAS Education has been very cooperative during the past three years in listening to our training needs and modifying some of the existing SAS classes to suit those needs.”

Westat also relies on a cadre of in-house SAS experts to teach beginner and intermediate SAS classes. “Once a year we have a SAS Institute speaker present at our Westat SAS user group meeting, and we invite speakers from outside organizations, such as the Federal Reserve Board and the Bureau of Labor Statistics,” said Raithel.

The education that Westat SAS users receive is not all piped in. Many Westat SAS users are experts in their own right. One former SUGI conference chair, two NESUG and one SESUG conference chair came from Westat’s staff. At Westat, SAS conference participation is indispensable. “We send a lot of people to SAS user group meetings” said Raithel. “During the past 13 years, Westat staff has published 200 SAS conference papers.

“We feel that conference experience is invaluable,” he said. “When our staff attends a SAS conference, they pick up tips from the presentations and a lot of good information comes back to Westat.

“After every conference, those who attend are on the hook for a 5- to10-minute presentation of the best papers and new ideas. We know that when we send people they’re going to enrich themselves by gaining new SAS knowledge, and that knowledge is going to come back and be available to other Westat staff.”

You can find some of Michael Raithel’s books, interviews, tips and past presentations on support.sas.com. You can meet Raithel at SAS Global Forum 2010 at the authors’ reception in the SAS Publishing demo area on Monday evening from 6 p.m. until 7:30 p.m.

Add “It’s not easy being a SAS programmer” to your SAS Global Forum agenda. This SAS Global Forum Tuesday lunchtime presentation by Raithel will take a lighthearted look at some of the societal, workplace and industry issues that SAS programmers routinely face. *An extra fee event.

The SAS Dummy: SAS: The . Manual

Wed, 03 Mar 2010 13:32:37 +0000

Check out AnnMaria's blog covering what she's learned by being on the bleeding edge: using SAS 9.2 for Windows x64 on a Win7 virtual machine. Also, her short primer on XPT files makes me want to add some native capability into SAS Enterprise Guide.

If your goal is to send a SAS data set into a new JMP session, you can use this task right now. In SAS Enterprise Guide 4.3 (later this year), we'll have a built-in Send To JMP feature.

SAS from Out in Left Field: SaviDataSet 1.0.0.5 Released

Wed, 03 Mar 2010 13:27:23 +0000

I just posted the latest version of the SaviDataSet reader/writer. The latest version allows for its use from the command line for converting SAS datasets to delimited, XML, and Excel files (2003 & 2007).

Please see the Help File in the Savian--> SaviDataSet --> Help.txt folder for instructions.

I will be incorporating a write feature in at some point but it is more complex since variables need to be defined.

The latest release can be found here:

SaviDataSet 1.0.0.5

the SAS-BI blog: Addressing Security at the Root - BI Dashboard 4.2 Tips

Wed, 03 Mar 2010 09:01:00 +0000

You may choose to address security of who can view what dashboards within a SAS Information Delivery Portal page by managing the 'Sharing' capabilities at the Page or Portlet levels, however the most gradular level might also require a 'lock down' to avoid users surfing for and locating dashboards they shouldn't see.

In order to accomplish this, utilize the 'Folders' tab in SAS Management Console and navigate to:

System->Applications->SAS BI Dashboard -> SAS BI Dashboard 4.2

IMPORTANT NOTE: You will need to modify each element individually as the current version of BI Dashboard (9.2M2 - 4.2) doesn't support moving the elements into an organized folder structure.

Jurassic SAS in the BI/EBI World: HTML, PDF, RTF (and more) all at the same time.

Wed, 03 Mar 2010 07:10:00 +0000

or . . . when is a session not really a session.

In my last post I illustrated explicity creating and using a session. Of course as with all things SAS related, that is not the only way to create a session!

In order for ODS to support creating html page with mixed text and graphics (e.g., a PROC PRINT and a PROC GCHART), ODS uses lightweight sessions. These are sessions without a lot of the overhead of a full session, but which allow a program to write some content, e.g., in the case of ODS, perhaps a graph where the generated HTML contains a link that replays the graph. So while it looks to the user like they are getting a table and graph in one request, there are actually two requests. Run this sample Table and Graph and take a look at the generated HTML for the img tag.

Note that this example is actually a SAS/IntrNet sample from my SAS Press Book . . . . . . but the Stored Process Server works the same way!

The generated link uses the replay program to do that. Upon a review of the generated HTML (just do a View Source in your browser) you see that the replayed graph is actually an entry in a SAS catalog. Well, it turns out that whenever you submit a request for the Stored Process Server to run a program, SAS creates a catalog and the catalog name is available to your program as the macro variable reference &_tmpcat.

And now it gets even more interesting because simply by writing something to that catalog causes SAS to create the lightweight session.

Now lets consider the case where we want to generate an HTML report, but you also want to provide a way for a user to access a printable version of the report (e.g., a PDF file) or an easily editable version (e.g., an RTF file) without having to rerun the reporting code. A lightweight session can be used to so that ODS will write PDF and RTF versions to the &_tmpcat catalog while generating/streaming the HTML version back to the users browser.

Just include statements like this in your program:

  filename pd catalog "&_tmpcat..shoes.pdf";
  filename rt catalog "&_tmpcat..shoes.rtf";

and then use the stpBegin macro for the HTML version and explicit ODS statements for the PDF and RTF versions:

  %stpBegin
  ods pdf file = pd style = sasweb notoc;
  ods rtf file = rt style=sasweb;

then include whatever your reporting code is (e.g., PRINT, TABULATE, REPORT, etc.).

We now need to include some HTML to provide the hyperlinks to that will replay the PDF and/or RTF versions. The following DATA steps creates macro variables whose values are the needed HTML:

  data _null_;
   call symput('replayPDF','''<center><a href="'||&_replay||'shoes.pdf">Printable (PDF)</a></center>''');
   call symput('replayRTF','''<center><a href="'||&_replay||'shoes.rtf">Editable (RTF)</a></center>''');
  run;

and, yea, I know the quoting is UGLY. I promise I'll blog about better ways to do this at some point in the future ;-).

and these ODS HTML statements generate the needed links in the HTML version.

  ods html text = &replayPDF
  ods html text = &replayRTF

And, as a bonus, since the ODS HTML statement does not write any content to the PDF or ODS destinations, those versions of the reports do not contain the hyperlink text!

And since the %stpEnd macro generates an

  ODS _ALL_ close;

statement, it will close all of the output destinations for you.

Feel free to check out this sample on my server.

And, of course, there are more examples of sessions and lightweight sessions in Building Web Applications with SAS/IntrNet®: A Guide to the Application Dispatcher. And they work pretty much the same way in the Stored Process Server (with the exception of the one issue highlighted in my PRIOR POSTING).

Happy Sessioning ;-).

NOTE: The blog of RTSL.eu: Project Plans in Excel - Grouping Tasks

Tue, 02 Mar 2010 20:20:51 +0000

In the two previous posts in this series I described how to create a neat and simple Gantt chart and how to add dates to the day numbers. In this post I’ll describe how to groups your tasks in the chart that was featured in the previous post. The picture alongside (right) shows the end result from today's post. Grouping tasks is a generally useful thing to do, but I also find that my list of tasks increases as time goes by, so I might not need groups to begin with, but they become a useful way of keeping my plan tidy after it has grown.

As with the previous cases, I’m going to describe a quick and simple method. The objective is to have a useful and communicative chart without spending too long on creating it and without making it difficult to maintain. We start with the chart that was created in the last posting (shown to the left). Remember my comments in the first post in this series: I expect SAS developers to run their own (small to medium sized) projects from time-to-time, and I expect them to know how to work to a plan.

Let’s begin by adding a group column (inserted as column A, and then filled with appropriate values):

Select cells A2 to E6, i.e. the cells that have the column headers and our data. Launch the Subtotals window (in Excel 2002 it’s available from the Data menu). Use ticks to add subtotals to Start, Effort and End (in the summary rows we want to see summarised values for these three columns - yes, I know it's not appropriate to "sum" Start and End, just bear with me!):

When you click OK you’ll see that outlines have been added, and Start, Effort and End are being totalled (and sub-totalled). We need to change the “(sub-)totals” for start and end to show minimum and maximum instead (see, I asked you to bear with me!). Select column C and use Edit/Replace to replace “SUBTOTAL(9,” with “SUBTOTAL(5,”; and in column E, replace “SUBTOTAL(9,” with “SUBTOTAL(4,”. Finally, you might want to select column A and remove the word Total. The result looks like this:

Q: What was that stuff with changing subtotal 9 to subtotal 5 and 4?

A: Excel's subtotal function does more than subtotaling. The first parameter is a numeric code that tells the function what you want it to do. 9 indicates summarisation, i.e. adding-up, but 5 indicates you want to see the minimum value, and 4 indicates maximum value.

To finish our work we might a) Replace “Grand” in cell A9 with something more suitable such as “TOTAL”, and b) enhance our conditional formatting to make the (sub-)total rows look distinctive. We can spot them because the task name is blank in these rows. Thus we change the conditional formatting as shown below for cell G3 as an example (then just use Format Painter to spread it across the whole Gantt chart). Note the additional clause in the AND function of the first clause in addition to the extra condition:

And here’s the final result:

This series of three posts has shown how to quickly and easily create a simple Gantt chart. My objective has been to produce a neat, communicative chart without too much effort. In addition, the chart has to be robust and allow me to make changes (such as adding tasks or changing them) without disturbing the chart and requiring me to start from scratch. These charts demonstrate that the developer is in control of their project (without spending more time on planning than on delivering the goods).

If I'm working on a large enough project I sometimes add a couple of embellishments to the chart (given that the size of the project merits the extra effort involved). I like to highlight "today" and I like to track %done and show it on the pseudo-bars of the Gantt chart. I'll describe how to do this in a bonus post next week.

It is possible to use Excel charts to create a similar output. The chart can look more pretty than what we've achieved above, but I find that it can become unwieldy and difficult to read & print as the number of tasks grows. Hence, I choose to use the worksheet rather than a chart. I'll describe how to do this in a bonus post in a subsequent week.

NOTE: The blog of RTSL.eu: NOTE: Data Set Reader and Log Analyser from Oceanview

Tue, 02 Mar 2010 20:08:17 +0000

UK-based Oceanview Consultancy have a couple of neat SAS utilities that just might meet your SAS team's needs. DSREAD reads a SAS data set and provides information about it without requiring SA software; and Elvis(!) is a log analyser par excellence.

Back in May 2006 when NOTE: was an email newsletter and boasted 3,700 subscribers worldwide, issue 17 featured the Elvis log analyser (and some awful usage of some of The King's record titles). Well, Chris Long wrote to me recently to tell me of his latest creation - DSREAD.

Chris describes DSREAD as a simple Windows command-line utility for working with data sets (SAS7BDAT ). The current version of DSREAD allows you to:

List the summary details of datasets
List the contents (the variable names and other attributes*) of datasets
Convert a dataset to CSV format for import into Excel and other software

The latter functionality could be useful in an automated system that uses a variety of technologies.

Chris lists a few limitations that you should read before downloading and using it. However, in a development environment it could prove useful.

It's good to see continued activity from Oceanview in the area of third-party SAS utilities (supported or otherwise). There are not many third-party SAS-related products available (the excellent Futrix being the notable exception).

With regard to Elvis, as issue 17 of the NOTE: email newsletter told us, it provides a novel and powerful environment for exploring your SAS logs. See the Elvis microsite for further detail.

Finally, if you like quizzes, you might like Chris's pet project: QuizBang. Visit the QuizBang web site, and see for yourself.

Key Happenings at support.sas.com: Referring to SAS and its products

Tue, 02 Mar 2010 19:10:51 +0000

Q: How do I refer to SAS in a scientific paper?
A: Always write the name "SAS" in uppercase letters with no periods.

As you might guess, there are more guidelines and specifics about how to refer to SAS and its products and services. The complete list of guidelines is available on our corporate Web site under the Press Center in Editorial Guidelines.

I have selected a few items from the guidelines that may more accurately address the question that was intended.

Do not use a trademark symbol after the word SAS when referring to SAS Institute Inc.

Include the SAS trademark notice shown below in a footnote at the bottom of the page or the end of the article.

SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration.

Scientific journals often require enough information to replicate your data findings. In these cases, the proper citation would be as follows (brackets "[]" indicate data that should be supplied by you):

The [output/code/data analysis] for this paper was generated using [SAS/STAT] software, Version [8] of the SAS System for [Unix]. Copyright © [year of copyright] SAS Institute Inc. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc., Cary, NC, USA.

Remember that I have provided only a partial list. Visit the Editorial Guidelines for full details.

NOTE: The blog of RTSL.eu: NOTE: Don Henderson Has a Blog

Tue, 02 Mar 2010 12:00:06 +0000

Somewhat humbly titled "Jurassic SAS in the BI/EBI World", long-time guru of SAS Don Henderson has joined the blogosphere. Don's CV is long and impressive and includes over 30+ plus years of using SAS;
Don is author of the SAS Press book "Building Web Applications with SAS/IntrNet: A Guide to the Application Dispatcher", and one of the principal architects for sasCommunity.org.

I've already learned useful stuff from Don's postings, including his January 26 post "An autoexec facility for the Stored Process Server" wherein he mentions that stored processes can have a program automatically run after each invocation (like a retrospective autoexec). I'd overlooked this useful feature.

JMP Blog: Credit Cards: A Method Behind the Madness

Mon, 01 Mar 2010 14:00:00 +0000

It’s no surprise that a Google search for credit card offers yields more than 100 pages of links (and ads). Lou Valente, one of our design of experiments experts, recently used JMP to share a direct marketing DOE case study from the 2007 book Testing 1 - 2 - 3: Experimental Design with Applications in Marketing and Service Operations by Johannes Ledolter and Arthur Swersey.

A financial institution sought to sell more credit cards and to maximize return on new cards. Which fees and interest rates would yield the most attractive product to new customers? The institution considered offering products with a lower annual fee, a new fee for opening the account, a lower initial interest rate and/or a higher long-term interest rate.

They planned to mail 120,000 offers in the first pass and then to market the most attractive offer aggressively. Using JMP, Lou created a full 2⁴ factorial experiment to show how they identified the 16 combinations of fees and interest rates that they would offer. The institution outlined each offer in a separate invitation letter and sent each letter to 7500 prospects.

Which of the 120,000 offers got the highest response rates? Lou initially used simple Distributions in JMP to interactively investigate the relationship of the factors for the highest response rate. The institution learned that lower annual lower fee, no opening fee and a lower longer interest fee were associated with the highest number of offer acceptances. They were surprised to learn that initial interest was not as important.

Lou then built a model. Clearly those in the business of offering credit cards wanted to determine the highest acceptable annual fee and maximum initial interest the new customers could bear. Later, they will want to examine cancellations, delinquencies and profit from these new customers. Having that data would be delicious food for JMP junkies and could be helpful to each of us when considering new credit card offers.

Interested in more DOE? Join Lou's March 15 live Webcast, Design of Experiments to Solve Expensive Process Problems. Registration is required.

the SAS-BI blog: Quick Preview of Gauges and Additional Settings Available in BI Dashboard (4.2)

Mon, 01 Mar 2010 09:00:13 +0000

Would you like to get a quick preview of the gauge you are using in the BI Dashboard Indicator? Just tap the 'Select ...' button to the right of the 'Gauge type:' dropdown box.

There are then images of what these gauges look like, such as the 'Dials' category below:

Also, if you select the 'Dynamic Gauges' category you will be able to edit the height/width of the gauge, display in 3D, the data format of the associated measure, and whether to orient the gauge horizontally or vertically:

SAS from Out in Left Field: SaviDataSet Alpha 1.0 On the Web

Mon, 01 Mar 2010 07:24:17 +0000

I am finally ready with my SAS dataset reader/writer for .NET. It is written in 100% managed code using .NET 3.5. The dlls can be found
here.

A sample .NET console application can be found in the program file entries after the installation.

Update 2/13/2010:

I am now testing this against a real-world project so I am finding little bugs here and there. These are being addressed for version 1.0.0.1. If you need a build sooner than it is released, let me know.

I have also found that you need at around 100+ observations for this to work correctly. I am investigating but keep that in mind while testing.

[LATE BREAKING] I fixed the obs issue so I went ahead and uploaded the fixed version. Watch for breaking changes which are in the Readme.txt file.

Update 2/29/2010

Based upon Chris' work, I added in a console application that exposed my existing work. It allows for export of the sas7bdat to Excel, delimited, and XML. I will add in import at some point and have designed the interface as such. For example, I put the parms in an XML file to provide enough flexibility to accomplish everything. That has pros and cons but I figured the pros outweighed the cons.

I am also doing something I have meant to do for a long time which is to build a data viewer. Hence, I don't want an interim release but will bundle it all together. I have spent some time this weekend starting that process. All of the components are here but I have to pull them together which I am hoping to do this week.

SAS Programming: AUC calculation using Wilcoxon Rank Sum Test

Mon, 01 Mar 2010 03:02:23 +0000

Accurately Calculate AUC (Area Under the Curve) in SAS for a binary classifier rank ordered data

In order to calculate AUC for a given SAS data set that is already rank ordered by a binary classifier (such as linear logistic regression), where we have the binary outcome Y and rank order measurement P_0 or P_1 (for class 0 and 1 respectively), we can use PROC NPAR1WAY to obtain Wilcoxon Rank Sum statistics and from there we are able to obtain accurate measurement of AUC for this given data.

The relationship between AUC and Wilcoxon Rank Sum test statistics is: AUC = (W-W0)/(N1*N0)+0.5 where N1 and N0 are the frequency of class 1 and 0, and W0 is the Expected Sum of Ranks under H0: Randomly ordered, and W is the Wilcoxon Rank Sums.

In one application example shown below, PROC LOGISTIC reports c=0.911960, while this method calculates it as AUC=0.9119491555


data test;
  do i = 1 to 10000;
     x = ranuni(1);
     if x + rannor(1) * 0.2 &gt; 0.7 then y = 1;
     else y = 0;
     output;
  end;
run;

ods select none;
ods output Association=Asso;
proc logistic data = test ;
    model y = x;
    score data = test out = predicted;
run;

ods select all;
data _null_;
    set Asso;
    if label2='c' then put nValue2=;
run;

%let Class=0;
ods select none;
ods output WilcoxonScores=WilcoxonScore;
ods output WilcoxonTest=WilcoxonTest;
proc npar1way wilcoxon data=predicted ;
     where y^=.;
     class y;
     var p_&Class;
     output out=wilcoxon;
run;

ods select all;
data AUC;
    set WilcoxonScore end=eof;
    array _A{2, 3} _temporary_;
    if Class=0 then do;
       _A[1,1]=N; _A[1, 2]=SumOfScores; _A[1, 3]=ExpectedSum;
    end;
    else do;
       _A[2,1]=N; _A[2, 2]=SumOfScores; _A[2, 3]=ExpectedSum;
    end;
    if eof then do;
       AUC=(_A[&Class+1, 2]-_A[&Class+1, 3])/(_A[1,1]*_A[2,1])+0.5;
       keep AUC; output;
    end;
run;

NPAR1WAY approach gets AUC=0.9119491555;
LOGISTIC reports c-statistic = 0.91160

SAS Programming: Partial Least Square

Mon, 01 Mar 2010 03:01:28 +0000

In some predictive modelling projects, we may have variables that most of the observations have the same value, while the small percentage rest ones are populated with meaningful values. For example, 90% observations have values=0 but the rest 10% have value=1, 2, 3.....for a variable called WebHits, etc. We may have a large number of such variables, say web hits at different pages, and due to the small percentage of differently valued observations, each variable show minimal predictive power.

But we have a large number of such variables, and a quick way to figure out whether they collectively show up predictive power, we may use Partial Least Square method.

In SAS, we can get the PLS scores and score new data in this way:


ods select none;
ods output XWeights=Xweights;
ods output XVariableCenScale=XCenScale;
proc pls data=model_sample   method=SIMPLS  details censcale varscale;
        model  Event = &covars;
        
/* this is for comparison purpose*/
        output out=pls_tomvars(keep=PRSPCT_ID Enroll &covars _xscr_:) xscore=_xscr_;
run;
ods select all;


proc transpose data=XCenScale out=XCenScale_t;
        id variable;
run;

data XCenScale_t;
   if _n_=1 then _TYPE_='MEAN';
   else _TYPE_='STD';
   set XCenSCale_t;
run;

data XWeights;
       retain _TYPE_ 'PARMS';
      set Xweights;
      _NAME_=compress('_xscr_'||_n_);
run;


data Xscore/view=Xscore;
       set XCenScale_t XWeights;
run;

proc score data=model_sample  score=Xscore type=PARMS out=epsilon_std;
      var &covars;
run;

Lightweight yet detailed explanation of PLS and its application to data mining projects can be found at:
Pharmaceutical Statistics using SAS: A Practical Guide by Alex Dmitrienko, Christy Chuang-Stein, Ralph B. D'Agostino, SAS Publishing 2007

StudySAS Blog: MedDRA and WHODrug Coding | StudySAS Blog [Digg]

Fri, 26 Feb 2010 20:55:25 +0000

Though there are other Coding dictionaries available along with MedDRA (ex: COSTART and WHOART), MedDRA dictionary is typically used in the US for Adverse Events coding. AEMedDRA coding is not the SAS programmers work; it is usually done by medical coder or database programmer.

the SAS-BI blog: Improve Web Report Studio Performance by Carefully Choosing Processing Locations

Thu, 25 Feb 2010 09:00:10 +0000

Information Map Studio allows data managers the ability to join tables, create new data elements, and label/format data items for business consumption. However due to this flexibility users must weigh the benefits of customizing information maps against the performance impacts of completing many of these tasks within the Information Map itself.

From anecdotal evidence (in 3.1), I have seen information maps successfully join small tables and display a few custom measures. However, when large tables are joined or a large number of custom measures are utilized, the performance (within Web Report Studio at least) can be dramatically impacted. (Note - I have found that running a test in Information Map for these cases is not indicative of the performance in Web Report Studio.)

So, once all the Web Reports are created and the performance improvements are required, how (in 3.1) can we switch where the custom measure is derived (from an expression within Information Map Studio to a data item in the source table/cube) without crashing the Web Reports?

Two options:

You can add these items and modify the associated web reports by hand - or
You can modify the XML to ensure that all the web reports use the new item instead.

Add the new measures into a copy of the original imap. Save out the xml & copy the new data element items from within the <ExternalResourceList ...> tagged area (located around the end of the xml file). Now in the original imap, export the XML and copy these new measures within the <ExternalResourceList ...> tagged area.

Then to switch the dataitem from a calculated expression to one of these new data elements follow the example below.

Included is what a simple calculated expression looks like in the information map XML:
<expr>
<resourceawarestrexpr exprtype="2" scope="2">
<reposobjuse prid="XR1">
<string strvalue="/">
<reposobjuse prid="XR2">
</resourceawarestrexpr>
</expr>

In order to 'trick' the information map into using a different data element, you need to change the entire xml block below into the following:
<Expr>
<ReposObjUse prid="XR3" />
</Expr>

Where the XR3 corresponds to the new data element item in the <ExternalResourceList ...> tagged area added earlier.

Of course - ALWAYS take a backup of the Information Map (and all associated WRS reports) since XML editing can introduce it's own problems.

NOTE: The blog of RTSL.eu: NOTE: Successes and Tools

Wed, 24 Feb 2010 22:25:41 +0000

My previous post was our 100th and marks quite a milestone. Since starting in July last year, Google Feedburner tells us:

yesterday we achieved our highest ever number of subscribers: 156
we've clocked-up 36,000 views
our most popular post of all time is Dashboards & Scorecards: What To Measure with 1,055 views (note that the widget in our right-margin shows popularity over the last 7 days)

And Site Meter tells us that the average visit length is over 2 minutes, so we know that you're reading the articles as well as just looking at them ;)

The NOTE: blog was preceded by the NOTE: email newsletter, the archive of which is still available from our web site. As I reflected in my December What's It All About? post, this blog endeavours to cover the range of experiences and knowledge that a professional SAS developer might be expected to encompass. In other words, the blog includes hints and tips on SAS syntax, but it also covers a range of other topics related to software development including:

Requirements capture
Design decomposition
Modelling
Construction techniques and development processes
Refactoring
Configuration management
Peer review
Dynamic testing
Effort estimation
Written and verbal communication
Interpersonal skills

Thank you for supporting the blog, and I'm glad you find it of interest.

In doing my work for RTSL.eu I find myself away from the office a lot. To keep up the pace of posts I need a small collection of gadgets and technology. My favourites are:

My Acer Aspire One that's small enough and light enough to take into any pub and restaurant in the evening, yet with a big enough screen (and enough computing power) to make it eminently usable
The Blogger blogging platform and associated Google services (described in Building the Blog) that provide such a range of flexible means of maintaining the blog and its posts, and keeping track of its usage
Morose Media's WMWiFiRouter that allows me to remain connected whenever I need to be
My trusty HTC HD2 mobile phone, running Windows Mobile and Morose Media's WMWiFiRouter!

And finally, a special mention is due for DropBox. It's like a memory stick in the internet cloud, and it keeps all of my blog stuff synchronised between laptop, office computer and any web terminal I may happen to use. I highly recommend it, and the first 2GB are free! It uses Amazon's S3 storage service (so it's reliable), it adds its own encrypted AES-256 layer (so it's secure), and reviews that compare it with other services (such as JungleDisk) regularly rate DropBox as top due to its user-friendly integration with Windows, Mac and Unix (and iPhone, but not Windows Mobile).

If you're interested in DropBox, use this link to get a free account and contribute an extra 250MB to my own DropBox. Given that NOTE: is free and offers no advertising, I hope you won't mind me offering this referral link!

NOTE: The blog of RTSL.eu: NOTE: Success Demonstrates BI Scope (the BI evolution)

Wed, 24 Feb 2010 21:45:51 +0000

Business Intelligence (BI) spans simple historic reporting to embedded real-time analytics. This is our 100th post and we're sharing our (minor) celebration with a SAS success.

BI is a commonly used term with a raft of different interpretations. Wikipedia begins to define it thus:

BI technologies provide historical, current, and predictive views of business operations. Common functions of Business Intelligence technologies are reporting, online analytical processing, analytics, data mining, business performance management, benchmarking, text mining, and predictive analytics.

There's a BI evolutionary path that starts with simple, static reporting on historic data (often delivered with spreadsheets) through to real-time predictive analytics embedded into front-office transactional systems. Many suppliers who claim to offer BI systems barely get off the ground on the BI flight to delivering real value to the enterprise.

All those products that offer sexy, shiny, slick graphics with animated 2.5D fuel gauges that make your historic data look exciting but don't begin to tell you about where you're headed are flattering to deceive. If you're considering implementing a BI solution, make sure your chosen software will give you the headroom to grow the value that the solution delivers. Don't box yourself in with a sexy solution that ultimately offers no real intelligence.

SAS salesmen will tell you that they can satisfy the full BI evolutionary path. And they'll point to their recent success in being awarded a Product of the Year award from Customer Interaction Solutions (CIS) magazine as evidence of offering solutions at the far end of the BI evolutionary path. In January, CIS magazine awarded a Product of the Year award to SAS Real-Time Decision Manager (RTDM).

SAS RTDM epitomises the ambition of true BI solutions. Incorporating business rules developed by the business users with a user-friendly graphical interface (and appropriate degrees of security), RTDM uses a service-oriented architecture to deliver SAS analytics into the heart of the customer interaction (telephone, web, ATM), promising recommendations, decisions and actions based upon the best information available.

The most popular reasons for considering a BI solution are to provide an analytical capability to more users, improve integration of data from a variety of sources, and to speed-up access to timely business data. Also, and by no means least amongst the reasons, businesses are often to keen to escape the bonds of spreadsheets and to gain a single, reliable & robust version of the truth.

In addition to the CIS magazine award, January saw SAS placed in the leaders' quadrant of the Gartner Magic Quadrant for Business Intelligence. Gartner's Magic Quadrant charts vendors' completeness of vision alongside those vendors' perceived ability to execute the vision. Gartner says that "leaders are vendors that are reasonably strong in the breadth and depth of their BI platform capabilities and can deliver on enterprisewide implementations that support a broad BI strategy."

Gartner's comments that are specific to SAS highlight SAS's focus on advanced analytics whilst noting that SAS customers use a broad range of SAS's BI capabilities. However, Gartner strikes a note of caution regarding the new challenges from big players like IBM and SAP who have recently bought previously-niche products such as SPSS, Cognos and Business Objects. Gartner suggest that SAS's comfortable position in the BI world may become less comfortable once the big boys get on top of their purchases.

I noted in December that SAS were aware of the changing market and felt up to the challenge. Given that SAS BI is currently in the Gartner leaders' quadrant and SAS Data Integration (DI) ranks equally highly in Gartner's view, SAS will consider themselves facing the challenge from a position of strength.

NOTE: The blog of RTSL.eu: NOTE: Issue 49 of VIEWS News is Available

Wed, 24 Feb 2010 12:05:00 +0000

Phil Holland has published issue 49 of VIEWS News, the quarterly journal of the VIEWS International SAS Programming Community. This quarter's issue includes graphical impact, encryption algorithms & format merges, and Excel functions available in SAS 9.2, plus a prize draw especially for VIEWS News readers.

You can view the 48 back-issues in the archive.

NOTE: The blog of RTSL.eu: NOTE: PROC MEANS Gives You All It's Got (and More!)

Wed, 24 Feb 2010 12:00:00 +0000

An oft overlooked parameter for PROC MEANS (and PROC SUMMARY) is COMPLETETYPES. It tells MEANS to create all possible combinations of the values of the classification variables, even if some of those combinations don't exist in the data. And PRELOADFMT will create combinations from values that don't even exist in your input data. This can be very useful in presenting what appears to be a more complete picture of the input data and can be equally useful in presenting a consistent layout amongst a group of reports (or regularly produced reports).

Here's a simple code example:

data sales;
  region = 'North'; product = 'Widget'; sales = 500; OUTPUT;
  region = 'North'; product = 'Foobar'; sales = 300; OUTPUT;
  region = 'South'; product = 'Widget'; sales = 100; OUTPUT;
run;

proc means data=sales /*completetypes*/ sum;
  class region product;
run;

If we run it without COMPLETETYPES we get:

     Analysis Variable : sales

region    product    Obs             Sum
----------------------------------------
North     Foobar       1     300.0000000
   Widget       1     500.0000000
South     Widget       1     100.0000000
----------------------------------------

And if we run it with COMPLETETYPES, we get (with the new information highlighted in red):

     Analysis Variable : sales

region    product    Obs             Sum
----------------------------------------
North     Foobar       1     300.0000000
   Widget       1     500.0000000
South     Foobar       0               .
   Widget       1     100.0000000
----------------------------------------

If there's a chance that your data won't contain all possible values for any given classification variable (this is a possibility when you regularly run reports on differing data), you might want to consider PRELOADFMT - it's a key word for the CLASS statement. The formats associated with the CLASS variables are used to tell MEANS the full list of values. See this example (key code highlighted in red):

proc format;
  value $regions 'North' = 'North'
   'South' = 'South'
   'West' = 'West'
   ;
run;

data sales;
  format region $regions.;
  region = 'North'; product = 'Widget'; sales = 500; OUTPUT;
  region = 'North'; product = 'Foobar'; sales = 300; OUTPUT;
  region = 'South'; product = 'Widget'; sales = 100; OUTPUT;
run;

proc means data=sales completetypes sum ;
  class region product / preloadfmt;
run;

We define a format for REGION which contains "West", but West is not in our data at all. We add PRELOADFMT as an option to the CLASS statement, and here's the output that we get (new bits in red):

     Analysis Variable : sales

region    product    Obs             Sum
----------------------------------------
North     Foobar       1     300.0000000
   Widget       1     500.0000000
South     Foobar       0               .
   Widget       1     100.0000000
West Foobar       0               .
   Widget 0               .
----------------------------------------

So, we've seen how we can zero-fill a data set using two different techniques, the first using all combinations of data within the input data set, and the second using a pattern of values from format(s).

See also:

PROC MEANS - CLASSDATA, MISSING
PROC FREQ - SPARSE

Sadly, the Summary Statistics task in Enterprise Guide (which uses PROC MEANS) does not offer COMPLETETYPES nor PRELOADFMT; but you can add them yourself within the Code Preview window.

NOTE: The blog of RTSL.eu: NOTE: More Online Training

Wed, 24 Feb 2010 11:35:00 +0000

Having mentioned Sunil's online training and featured his mooted courses in India yesterday, I have to mention Andrew Karp's ever growing catalog of online and face-to-face courses at Sierra Information Services. And the Virtual SAS Users Group who offer yet more online courses. Sitting alongside the online courses that SAS offer, there's a wealth of material waiting for you to sign-up and learn more about the technology and your industry.

JMP Blog: Creating a Simple Dashboard in JMP

Tue, 23 Feb 2010 18:43:37 +0000

Business users have been flocking to JMP for its data visualization and analytical capabilities. You can create a simple dashboard with a minimal amount of scripting. The functions that I will use are: New Window(), H List Box(), and V List Box().

The New Window() command creates a new report window where all of your analyses will be placed. H List Box () and V List Box() allow you to visually arrange analyses horizontally and vertically, respectively. Without specifying H List Box() and/or V List Box(), all analyses will appear stacked on top of each other.

Let’s build a dashboard step by step:

1. In a new script window, use the following code to open a JMP data table.

 MyDataTable = Open( "$SAMPLE_DATA/Big Class.jmp" );

I am using the Big Class data table found in the JMP sample data directory. However, your path may look something like this: “C:/My JMP Analyses/Favorite JMP Data.jmp”.

2. Create a new report window.

MyDataTable = Open( "$SAMPLE_DATA/Big Class.jmp" );

New Window( "Student Report",  );

The New Window() command requires a title, in this case “Student Report,” and a display box which is left empty for now.

3. Add an analysis to the report. I recommend using the point-and-click method to run an analysis and then take the JSL code that was written behind the scenes. Under the Hot Spot (red triangle) of your analysis report, select Script and then Copy Script. Add the script to your New Window() command as the display box. I’ve used the Bivariate Analysis.

MyDataTable = Open( "$SAMPLE_DATA/Big Class.jmp" );

New Window( "Student Report", Bivariate( Y( :weight ), X( :height ) ) );

4. Add additional analyses by making sure to end each analysis script with a semi-colon. I’ve added Tabulate.

MyDataTable = Open( "$SAMPLE_DATA/Big Class.jmp" );

New Window( "Student Report",  

	

	Bivariate( Y( :weight ), X( :height ) );

	Tabulate(

		Show Control Panel( 0 ),

		Add Table(

			Column Table( Analysis Columns( :height, :weight ), Statistics( Mean ) ),

			Row Table( Grouping Columns( :sex, :age ) )

		)

	);

	

);

5. String together analyses horizontally by using the H List Box() command. Display boxes are strung together using a comma: H List Box(Display Box 1, Display Box 2,…). Similarly, string analyses vertically by using V List Box(Display Box 1, Display Box 2,…);

 MyDataTable = Open( "$SAMPLE_DATA/Big Class.jmp" );

	

New Window( "Student Report",  



	H List Box(

	

		Bivariate( Y( :weight ), X( :height ) );

		

		Tabulate(

			Show Control Panel( 0 ),

			Add Table(

				Column Table( Analysis Columns( :height, :weight ), Statistics( Mean ) ),

				Row Table( Grouping Columns( :sex, :age ) )

			)

		);

	)

	

);

6. You'll find a sample dashboard with multiple analyses stacked horizontally and vertically in the JMP File Exchange under my name. Further dashboard customization, such as coloring data points and adding a title, are also found in the example.

the SAS-BI blog: The easiest way to generate additional log information for Web Report Studio (3.1 and 4.2)

Tue, 23 Feb 2010 13:34:28 +0000

Ran across a situation last week that proved extremely helpful. We needed to generate detailed logs on Web Report Studio use in order to verify pooling was setup correctly and then to monitor performance of some poor performers. The easiest and most straightforward way to accomplish this was to follow these steps.

1. Verify that the Tester is a member of the WRS Administrator group in SAS Management Console
2. Log into SAS Web Report Studio
3. Modify the URL to: /SASWebReportStudio/debugLog.do?LogAllActions=true and hit 'Enter'
4. You can also change the debug level by modifying the URL to the following format

/SASWebReportStudio/debugLog.do?LogName=log.context&LogLevel=level

An example:

/SASWebReportStudio/debugLog.do?LogName=com.sas.apps.citation&LogLevel=DEBUG

and hit 'Enter'
5. Continue with testing.
6. Don't forget to turn off logging before getting out of Web Report Studio

/SASWebReportStudio/debugLog.do?LogAllActions=false

7. Log off of WRS and retrieve the WebReportStudio.log file

For 3.1 - the reference material is available on pg 110 of http://support.sas.com/documentation/configuration/biwaag.pdf

For 4.2 - the same instructions apply. See the section 'Configure Debug Logging Dynamically' on page http://support.sas.com/documentation/cdl/en/biwaag/63059/HTML/default/a003050065.htm.

NOTE: The blog of RTSL.eu: What is a Project?

Tue, 23 Feb 2010 13:18:17 +0000

Alongside our series of posts on creating Gantt charts in Excel for the purpose of managing small to medium sized projects, a discussion on "what is a project?" might be useful. Most of us feel we understand the general usage of the term "project", but what does it mean in the context of Project Management?

The Cambridge Advanced Leaner's Dictionary defines the noun project as "a piece of planned work or an activity which is finished over a period of time and intended to achieve a particular aim". The key attributes are a) having an aim or purpose, and b) being able to define a start and end date/time. These attributes make projects distinct from ongoing operational, live, production or business as usual (BAU) activities.

The existence of the end date is an important element of the project, and a lot of work goes into agreeing the end date and then making sure the project is delivered/completed by the end date. Plans are drawn-up, often including Gantt charts.

Projects exist to create something new; something that hasn't been created before. So, we can have a project to create a new business intelligence solution for a new SAS customer, or we can have a project to create a new data mart. Producing a bunch of standard reports every week is not a project, but the task of producing the reports for an individual week might be considered a project because it has a defined aim and a defined end date.

An important element of project planning is to be sure that everybody understands what "finished" means, and what it looks like. Does your development project finish when your code is successfully tested and goes live, or will their be a "warranty period" wherein the development team will be on-hand to investigate and resolve problems quickly, or does all of the user training need to be delivered too?

It is the responsibility of everybody involved with the project to make themselves aware of the full set of deliverables and the dates by which they should be delivered. And it is every team member's responsibility to focus on achieving these deliveries on-time, on-budget, and with the expected degree of quality.

I should add that sometimes projects finish before they have delivered their aims. It is good practice to monitor progress throughout the life of the project, but it's also good practice to review whether the aim is still achievable and whether the expected benefits will still accrue as a result. If you have a multi-year project aimed at producing a new, improved widget yet (since you started your project) the world has discovered that foobars are preferable and everybody's getting rid of their widgets, you better give serious consideration to canning your project (or resetting its aims, or increasing the marketing budget for your widgets!).

NOTE: The blog of RTSL.eu: Project Plans in Excel - Adding Dates

Tue, 23 Feb 2010 13:17:03 +0000

In the previous post in this series I described how to use Conditional Formatting to create a neat and simple Gantt chart alongside a simple Excel-based project plan. In this post I’ll describe how to use dates in addition to the day numbers that were featured in the previous post. The picture alongside (right) shows the result from today's post.

As with the previous case, I’m going to describe a quick and simple method. This method also takes weekends into account as non-working days. We ended the last post with what you see alongside (left).

So, let’s begin by adding the date for day 1 into cell F1. I'm typing “22/2” to represent 22nd February). It’s not readable in the small width of the cell, so we’ll go to the Format Cells window (you can use Ctrl-1 to get there quickly) and select text orientation as 90 degrees. Then, to get the date format that we want, we’ll stay in the Format Cells window and specify a custom number format of “dd-mmm (ddd)”. If the height of row 1 doesn’t automatically increase for you, just do it manually. You should have a result like this:

Now, let’s set G2 as =F1+1 and copy the formatting from F1. This gives us the correct result for G2, but it’s not a sequence that we can ask Excel to follow because we’ll get weekends included, so we should set the formula for G2 to =IF(WEEKDAY(F1)<>6,F1+1,F1+3). In other words, if the previous day is not a Friday, just add 1 to the date; if it is Friday, add three to the date. We can copy and paste G2 across the remainder of the row:

So, with very little effort, we have a Gantt chart with day numbers and dates.

In one further posting I’ll describe how to add groups of tasks and how to summarise them.

NOTE: The blog of RTSL.eu: Project Plans in Excel (simple, quick and effective)

Tue, 23 Feb 2010 13:12:17 +0000

For any piece of work other than the smallest, it’s worth planning. Planning doesn’t have to mean creating a huge monster in Microsoft Project - I find that Microsoft Excel (or similar) is often sufficient (and a lot more accessible to the team). This post (and the series of posts that follow) describes how to quickly and efficiently create an adequate plan for small to medium sized projects.

I don't expect all developers to be expert project managers, but I do expect my team members to understand the role of the project manager, to know how to work to a plan, and to focus on delivery. And I do expect developers to run their own (small to medium sized) projects from time-to-time.

A project plan can consist of just a list of tasks (preferably with start and end dates) together with the name of the person who will complete the task, but this can be made to communicate a lot more if you can deliver a Gantt Chart too. The name “Gantt Chart” sounds challenging to anybody who hasn’t met one before, but actually it’s rather simple format that most people are familiar with (often without knowing the name). Gantt Charts can contain a lot of detail and embellishment, but I’m going to describe how to create a simple yet communicative chart very quickly.

To illustrate what to do, I’ll use Microsoft Excel 2002, conditional formatting, and formulae.

Let’s start by creating the basic list of tasks, their start dates, and their durations (I’ll ignore the name of the actionee in order to save space):

We can add a formula to calculate the end date: d3 set to =B3+C3-1, etc.

Next, let’s add columns for the days of our Gantt:

To create our Gantt, we want to create solid-coloured cells where the day number (in row 2) lies between or equal to the start and end days respectively. To do this we will use Conditional Formatting. From cell F3 we need the following Conditional Formatting (accessed from the Format menu in Excel 2002):

In addition, we’ve used the Format button to set the pattern for the cell shading to solid green. And, having clicked OK, we see that cell F3 turns green, reflecting the fact that activity #1 is to be performed on day 1.

We now need to copy this formatting to all of the other cells of the Gantt, but if we copy and paste we’ll get the wrong results because the cell references will be move relative to the point where we paste the formatting. So, we need to change the formula to tell Excel which bits of the formula should move when we copy/paste, and which bits should be fixed. The modified formula is: =IF(AND($B3<=F$2,F$2<=$D3),1,0)

Armed with the correct formula, we can use the format painter to copy the formatting, or we can use Copy and then Paste Special. Either way, having done so we will see the end result:

That’s a simple, quick, but communicative Gantt chart. In subsequent posts I’ll describe how to quickly translate day numbers into dates, how to add grouping of tasks, and how to indicate progress (% done).

What I have described is just one of many routes to the same end. There are plenty of resources on the web to guide you to a similar result (or better) with Microsoft software and with others. Plus, SAS/OR offers plenty of project management functionality, including Gantt charts from PROC GANTT.

NOTE: The blog of RTSL.eu: NOTE: New Course in India: Best Practices in SAS Statistical Programming for Regulatory Submission

Tue, 23 Feb 2010 11:30:00 +0000

Having successfully conducted his long-titled course in the USA, Netherlands, and online, my friend Sunil Gupta is considering the possibility of running it in one or two cities in India. While the schedule is not yet finalised, the favoured period for the two-day class is around April end/early May in Bangalore and maybe Hyderabad.

For further details of the course, see the overview of the online version. To register interest in the Indian course(s), email Sunil directly (and tell him NOTE: sent you!).

You might also like to know that Sunil is presenting his "Preparing SAS Programmers for the Pharmaceutical Industry (An Introduction)" course as a pre-conference course at SAS Global Forum (SGF) 2010. There may still be places available.

The Princess of Science: ‫The Simpson‬

Tue, 23 Feb 2010 10:09:04 +0000

‫במשחק השביעי והמכריע בסדרת גמר אליפות הכדורסל הארצית, התמודדה קבוצת ספרינגפילד בולס מול יריבתה המושבעת, קבוצת יוטה סופרגז. כצפוי, המשחק הוכרע על פי היכולות האישיות של כוכבי שתי הקבוצות: בארט מספרינגפילד ויוחנן מיוטה. במחצית הראשונה היו לבארט 40% אחוזי קליעה מהשדה, בעוד שיוחנן צלף ב- 50% מנסיונות הקליעה שלו. במחצית השניה צפינו בהתעלות אישית של [...]‬

Key Happenings at support.sas.com: SAS gets your attention with a text message!

Mon, 22 Feb 2010 19:01:42 +0000

Contributed by Scott Vodicka, a member of the SAS Global Consulting Business Intelligence Practice

An interesting topic came up the other day in one of my email conversations. What was the topic of the email? Well, I am glad you asked. It was how do we send text messages from SAS to someone's cell phone?

It is so easy that you will not even believe it! As a lot of us know SAS supports sending emails from a data step program, and the publish and subscribe model supports delivering information via email. The quick answer is to use the email address of the person's phone in the TO: field for the data step, or the email address used to subscribe to a channel.

So, now that the easy part is done, let me give you some information that will help you solve the problem I know you are now facing: How do I determine the users email address for their phone? There are a couple of tricks that make this really easy.

Below is a short list of popular carriers with their associated domains:
  AT&T:          @txt.att.net
  Alltel:           @message.alltel.com
  Sprint:           @messaging.sprintpcs.com
  T-Mobile:         @tmomail.net
  Verizon:          @vtext.com
  Virgin Mobile:     @vmobl.com

Check out these web sites for updated information on carriers, especially since this sector is constantly in flux.

http://en.wikipedia.org/wiki/List_of_carriers_providing_SMS_transit

http://edgedirector.com/htm/page.htm

If your phone number is: (919) 555-1234 (not a valid number), and your carrier is AT&T then the email address for your phone is: 9195551234@txt.att.net

Another way to find out your phone's address is to send a text message from your phone to your SAS email account: and you’ll get the email address for your phone in the from field (it might go to your junk folder).

Here is a sample data step program that sends an alert as a text message to a user's cell phone. Before you submit the program below be sure to update your SAS configuration file for the following options:
EMAILSYS EMAILHOST EMAILID

You may have to supply the EMAILPW value depending on how your SMTP server is configured. You may specify the EMAILID and EMAILPW values in your SAS program via an options statement. See SAS documentation for details.

%let email_alert="9195551234@txt.att.net";

%let email_from="SAS Monitoring System ";



%let errormsg=Server failed to start;

%let host=metadata_server;



filename em_out email to=(&email_alert) from=(&email_from) 

    subject="Alert: SAS 9.2 Critical Status Error";

data _null_;

  file em_out;

  format timechar $21.;

  timechar=put(datetime(),datetime21.2);

  put timechar "ERROR : &sysuserid.@&host - &errormsg..";

run;

Important Note: Many people are charged per text message. Be sure to get permission before sending a text message to an individual.

SAS Global Forum: “Enormous Power at my Fingertips”

Mon, 22 Feb 2010 14:43:00 +0000

Last month I pointed you towards the conference t-shirt contest on sasCommunity.org. The designs and coding techniques on display were quite creative, and the contest triggered some fun conversation among community members. Now the votes have been cast.

And the winner is …self-proclaimed “new kid” Lynne Krajevski. I loved what she had to say about her unique design: “SAS is enormous power at my fingertips, always within my reach - this is what I was trying to convey with my design.” Way to go Lynne!

How can you get one of these limited number t-shirts? Come to SAS Global Forum 2010 in Seattle to find out!

JMP Blog: Saving Graphs, Tables and Reports in JMP

Mon, 22 Feb 2010 14:24:00 +0000

Here are some simple ways to save both tabular output and/or graphs in JMP.

When you copy information from JMP, you can either use “Copy” from the Edit menu or the tool bar. The copied information is copied to an active windows clip board.

1. If you want to save the whole report output with graphs, simply choose Edit/Copy to store the active window to the clipboard and then paste the results in your document.

2. You can save just a portion of the results by choosing the “fat-plus" tool from the toolbar (or Tools>Selection) and then clicking on the table, column or graph you would like save. You then copy and paste as before.

a. From any JMP output window, click on the selection tool in the toolbar (the fat plus)
b. Click on the content you’d like to copy – the selected content is highlighted. To extend a selection, hold the Shift key
c. Click Edit>Copy

d. Open the program where you’d like to paste the content and select Paste>Paste Special. From the list, select Picture (Enhanced Metafile).

By selecting Picture (Enhanced Metafile), you have the ability to grow and shrink the image, ungroup the image for manipulation. This options also takes up less storage space.

Once the image has been pasted in your application, if you right-click within the image and select Edit Picture, you then have the ability to manipulate that image.

3. JMP also provides methods that allow you to customize report output and save it as a graphic, text, Word, .rtf, or .html. The text format is the only format that doesn't save graphs. Select File>Save as

4. You can also save selected graphs or tables by choosing Edit>Save Selection As. This option only allows you to save the selected item as graphics with an extension .png, .jpg, .svg, .gif or .wmf

5. You cannot save output directly from the report; instead, you must duplicate it by selecting Edit>Journal or Edit>Layout. Both commands will reproduce the report in a new window and allow you to customize the look of the report. The layout option provides a few more editing options, such as allowing you to ungroup parts of a report. Select File/Save and choose a format above to save a journal or layout window.

To learn more about saving data tables, graphics and reports, refer to Chapter 4 of the User Guide in JMP: “Saving Data Reports and Sessions.”