18
Aug

Making the sas.servers script look pretty

If you are a SAS administrator managing an environment on UNIX or z/OS, you must use the sas.servers script on a regular basis. As you know, one of its uses is to display the current status of all servers. Is the output accurate? Absolutely. Is it easy to read? Relatively. Is it visually attractive? Not so much.

Human beings are visual creatures. Conventional wisdom says that a picture speaks a thousand words. However, I am not using images to improve the output, but another powerful tool: color. According to this Xerox paper, color captures attention and enhances productivity. It can improve search time, reduce errors, and increase comprehension. As a result, this blog post provides the steps for applying color and an easy-to-read layout to make the sas.servers script look cute and even fun!

Preliminary Steps

Before we begin, it is important to give you some recommendations:

  1. Stop all SAS services.
  2. Backup the files: sas.servers, sas.servers.mid and sas.servers.pre
  3. Apply these changes to a Development or Testing environment.

As a matter of fact, I consider steps 1 and 3 as optional. This script is only used by the SAS platform to start/stop/restart or check the status of the servers. Moreover, if you follow all steps carefully, you can apply these changes safely in a Production environment. In contrast, step 2 is relevant; it is always a good practice to backup essential files before modifying them. In case you need to rollback, you can restore them easily and quickly.

Things to Consider

The About the sas.servers Script section from the SAS 9.4 Intelligence Platform: System Administration Guide provides a caution message: "You should not directly update the sas.servers script." In our case, the type of customization we are about to perform requires a manual update. Don't worry, your script is in good hands.

Furthermore, if you ever need to update the sas.servers script because you want to add/remove a server, you have to run the generate_boot_scripts.sh script to regenerate this file. After doing so, you can lose all the changes made in this post. Keep this in mind, so you can backup the current files before attempting this task.

Easy-to-read Layout

Let's jump into the details. First of all, let's define the new layout of the desired output. Grab your favorite text editor. I use Notepad++ at the office, and Sublime Text at home. Go to your /SASCONFIG/Lev1/ directory and run a regular status command as the sas user. Assuming you followed the preliminary steps and depending on the products installed and the number of SASServers deployed, you should get a similar output:

./sas.servers status
SAS servers status:
SAS Web Infrastructure Data Server is NOT up
SAS Metadata Server 1 is NOT up
SAS Object Spawner 1 is NOT up
SAS DIP Job Runner 1 is NOT up
SAS Information Retrieval Studio Server is NOT up
SAS JMS Broker is NOT up
SAS Cache Locator Service ins_41415 is NOT up
SAS Web Server is NOT up
SAS Web App Server SASServer1_1 is NOT up
SAS Environment Manager is NOT up
SAS Environment Manager Agent is NOT up

Copy that output, except the first line, and paste it into your text editor. Here you can modify the output according to your taste. What I did was to identify the longest line, which is SAS Information Retrieval Studio Server, then I added four spaces to the right and substituted the legend is NOT up for [DOWN]. Likewise, I applied these changes to the rest of the servers, removed the instance number, and kept them all aligned:

SAS Web Infrastructure Data Server         [DOWN]
SAS Metadata Server                        [DOWN]
SAS Object Spawner                         [DOWN]
SAS DIP Job Runner                         [DOWN]
SAS Information Retrieval Studio Server    [DOWN]
SAS JMS Broker                             [DOWN]
SAS Cache Locator Service ins_41415        [DOWN]
SAS Web Server                             [DOWN]
SAS Web App Server SASServer1_1            [DOWN]
SAS Environment Manager                    [DOWN]
SAS Environment Manager Agent              [DOWN]

I chose the pair [ UP ]/[DOWN] to reflect the status because I wanted the look and feel from CentOS 6 boot process. If you are/were a CentOS 6 user, you can remember that the services display the legends: [ OK ] or [FAILED] when booting. You are free to use other alternatives such as ACTIVE/INACTIVE or perhaps RUNNING/STOPPED with or without brackets. Now that the layout is finished, let's move on to the color department.

All You Need Is Color

The fun finally arrived. Let's integrate the layout into our three scripts and add the main ingredient: color!

Again, I am assuming at this stage that you already backed up the files: sas.servers, sas.servers.mid, and sas.servers.pre. Next, open the sas.servers file with your vi editor and add this code anywhere at the top, below the block of comments:

#*****
# Custom Colors for status command
# RED   for [DOWN]
# GREEN for [ UP ]
# NC    for No Color
#*****
RED='e[31m'
GREEN='e[32m'
NC='e[0m'

These lines create three variables with three different color codes: a RED variable with code 31, a GREEN variable with code 32, and a NC variable with the default color code. The definition of these variables is optional but recommended, since they help you debug problems or change colors more easily. If you prefer to use different colors or attributes, you can play with the codes as shown in Bash tips: Colors and formatting.

Considering that the sas.servers script does not contain all servers, you have to add the same code to sas.servers.mid and sas.servers.pre files.

First Example

Now that we have defined the three variables in all the necessary files, let's use them. I'll show you how to apply them to a couple of servers, and then you can replicate it to the rest. The first one is the SAS Metadata Server. Open the sas.servers script again and find these lines:

# SAS Metadata Server
SASMETA_WONT_START_OTHERS="The remaining SAS servers will NOT be started as a result."
 
SASMETA1_IS_UP="SAS Metadata Server 1 is UP"
SASMETA2_IS_UP="SAS Metadata Server 2 is UP"
SASMETA3_IS_UP="SAS Metadata Server 3 is UP"
SASMETA4_IS_UP="SAS Metadata Server 4 is UP"
SASMETA5_IS_UP="SAS Metadata Server 5 is UP"
 
SASMETA1_IS_DOWN="SAS Metadata Server 1 is NOT up"
SASMETA2_IS_DOWN="SAS Metadata Server 2 is NOT up"
SASMETA3_IS_DOWN="SAS Metadata Server 3 is NOT up"
SASMETA4_IS_DOWN="SAS Metadata Server 4 is NOT up"
SASMETA5_IS_DOWN="SAS Metadata Server 5 is NOT up"

Since I have a single Metadata Server instance, the only meaningful variables are SASMETA1_IS_UP and SASMETA1_IS_DOWN. Delete their values, copy the correct string from your text editor, and paste it in both variables. Fix them accordingly:

SASMETA1_IS_UP="SAS Metadata Server                        [ UP ]"
#more instances
 
SASMETA1_IS_DOWN="SAS Metadata Server                        [DOWN]"
#more instances

The final touch is to give color to our script. Use the GREEN and RED variables for the UP/DOWN statuses. It is important to include the NC variable at the end to remove all attributes:

SASMETA1_IS_UP="SAS Metadata Server                        [ ${GREEN}UP${NC} ]"
#more instances
 
SASMETA1_IS_DOWN="SAS Metadata Server                        [${RED}DOWN${NC}]"
#more instances

Second Example

The process is the same for all the servers defined in the sas.servers script. For the other two scripts it is a little different, but still quite easy. I am going to use the SAS Web Infrastructure Data Server as the second example. Open the sas.servers.pre script and look for the server_status() function. Pay attention to these lines:

       if [ $? -eq 0 ]; then
          # Server is already running
          echo "SAS Web Infrastructure Data Server is UP"
       else
          echo "SAS Web Infrastructure Data Server is NOT up"
       fi
    }
    else
      echo "SAS Web Infrastructure Data Server is NOT up"

A subtle difference from the previous example is the echo command. In the sas.servers script, there is a logmsg() function that uses the "echo -e" command behind the scenes. In this case, we have to explicitly add the -e option to enable the interpretation of backslash escapes. Let's also integrate the color and layout:

          echo -e "SAS Web Infrastructure Data Server         [ ${GREEN}UP${NC} ]"
       else
          echo -e "SAS Web Infrastructure Data Server         [${RED}DOWN${NC}]"
       fi
    }
    else
      echo -e "SAS Web Infrastructure Data Server         [${RED}DOWN${NC}]"

At this point, with the above examples, you should have a solid idea about the required changes to accomplish our goal. Now it is your turn to apply them to the rest of the servers.

Finished Product

If you followed this article in detail and performed the steps in all the required servers, your output should resemble mine:

Get the green light by running the start command:

Final Thoughts

I am a visual person with a curious mind. One of the things I like is to customize the tools I use the most, so I decided to make the sas.servers script output a little more attractive to my eyes. I hope you liked the result. If you are still not sure whether to implement this idea or not, let's suppose there are some problems with a couple of servers in your environment and they stop running. Which output would you rather look at? Which one is easier to spot an issue? Let me know your thoughts in the comments below, or even better you can share your creative outputs!

Making the sas.servers script look pretty was published on SAS Users.

9
Feb

Middle Tier Changes and Upgrades in SAS 9.4 M4

Since the SAS 9.4 M2 release in December 2014, there have been several refinements and updates to the middle tier that are of interest to installers and administrators. In this blog, I’m going to summarize them for you. What I’m describing here is available in the newest SAS release (9.4 M4). I’ll describe them at a high level, and refer you to the documentation for details and how to implement some of these changes.

Security enhancements

Preserve your TLS Customizations:
For security purposes, many of you will manually add TLS configurations, either to the SAS Web Server, the SAS Web Application Server, or both. In addition, you may prefer to use your own reverse proxy server (such as IIS), either instead of, or in addition to, the SAS Web Server. Before the 9.4 M4 release, when upgrading or applying maintenance, you had to undo these custom configurations, perform the upgrade, and then apply the custom configurations again. Now, the upgrade will preserve them, making the process much easier. See Middle-Tier Security in the Middle Tier Administration Guide, Fourth Edition for full details.

Newer versions of OpenSSL are now provided (see doc for specific version numbers):
A Java upgrade enables enforcement of TLSv2. TLS is now considered the security standard for https connections, (SSL is obsolete) and this can be enforced with configurations to the SAS Web Server and the SAS Web Application Server. The new version of Java SAS is using (Ver 1.7+) now allows for this. One important thing to be aware of is that certificates are completely independent of which protocol you are using, and therefore any certificates you may have been using with SSL should work equally with newer TLS protocols.

Management of the trusted CA (Certificate Authority) bundle:
SAS now has a trusted CA bundle, that can be managed by the SAS Deployment Manager, in a new location:  SASHome/SASSecurityCertificateFramework/1.1/cacerts/. The CA certificates can be root certificates, intermediate certificates, or both. Here’s what the menu item looks like:

Middle Tier Changes and Upgrades in SAS 9.4 M4

Previously it was necessary to manually add your root/intermediate certificates to the Java truststore “cacerts,” located inside the JRE; now it’s done through the new interface. If you are on Windows, you must also add trusted CAs to the Windows store (as before), which will make them available to any browsers running there. This is documented at http://www.sqlservermart.com/HowTo/Windows_Import_Certificate.aspx and elsewhere online.

Security Support for SAS Web Applications – white list external sites, and HTTP request methods:
For added security, web sites hosting SAS web applications can now maintain a white list of external URLs that are allowed to connect in. This provides protection against Cross Site Request Forgeries, and other vulnerabilities. This is what the prompt looks like in the SDW:

Middle Tier Changes and Upgrades in SAS 9.4 M4

HTTP request methods can also be specified as allowed/not allowed. The list of URLs can be specified during installation in the SDW (shown above), or using the SAS Management Console. You can disable whitelist checking entirely, and you can add a “blacklist” or specific sites to always block. You can also block based on request method–ie, GET, POST, PUT, etc. See the Middle Tier Administration Guide for details.

Forward Proxy Configuration:
You can now set up SAS web applications to forward external URL requests through a proxy–here it’s called a forward proxy server. Many organizations do this behind their firewalls. See details for how to set this up in the administration guide.

Other miscellaneous changes:
As an administrator you can now force users to Log Off using SAS Web Administration Console.    You can also send emails to one or more users from the same window.  This is what the menu looks like:

Middle Tier Changes and Upgrades in SAS 9.4 M4

Faster start-up time for the SAS Web Application Server

JMS Broker (ActiveMQ) now uses Version 5.12.2 (fixed bugs).

SAS Web Server now uses version 5.5.2 and includes an updated mod_proxy_connect module for TLS tunneling.

References

SAS 9.4 Intelligence Platform: Middle Tier Administration Guide, Fourth Edition

Encryption in SAS 9.4, Sixth Edition

 

tags: SAS 9.4, SAS Administrators, SAS Professional Services, security

Middle Tier Changes and Upgrades in SAS 9.4 M4 was published on SAS Users.

20
Jan

Comparing SAS/GRAPH® 9.4 capabilities with SAS/GRAPH® Version 6

SAS/GRAPH 9.4 capabilitiesI remember my grandparents talking about how hard things were for them growing up. They would say, “Things were so bad that we had to walk uphill, both ways, in the freezing snow to get to school.” It was always hard for me to relate to these statements because the school bus picked me up at the end of my driveway. Fast forward to today and people are riding on hoverboards. Through the years, advancement in transportation has made it easier for us to get where we need to be.

SAS/GRAPH® Version 6

The evolution of SAS/GRAPH® is similar. In the earlier days of SAS® software, during Version 5 and 6 of SAS/GRAPH, I understood how difficult it was to create some of the graphs that customers wanted. A customer recently asked whether I could send him the code that produced the graph below, which he found in the SAS/GRAPH® Software: Reference, Volume 1 Version 6 Edition:

sasgraph-9-4-capabilities

In Version 6, the only way to create this graph, referred to as a butterfly chart, was by using SAS/GRAPH and the Annotate facility. The annotation statements added over 60 lines of code to the program.

Below is a snippet of the Version 6 program that created the bars on the left side of the graph.

Click this link to see the entire Version 6 program.

     /* female bars on left */
    %bar(39.8, 10.5, 25.0, 20.0, blue, 0, solid);
    %bar(39.8, 20.71, 15.0, 30.7, green, 0, solid);
    %bar(39.8, 31.42, 10.0, 41.42, red, 0, solid);
    %bar(39.8, 42.14, 32.0, 52.14, blue, 0, solid);
    %bar(39.8, 52.85, 33.0, 62.85, green, 0, solid);
    %bar(39.8, 63.57, 36.0, 73.57, red, 0, solid);
    %bar(39.8, 74.28, 35.0, 84.28, blue, 0, solid);
    %bar(39.8, 85.0, 33.0, 95.0, green, 0, solid);

SAS/GRAPH® 9.4

Fast forward to SAS® 9.4. ODS Graphics and SG procedures have been part of Base SAS® since SAS® 9.3, making it much easier to create high-quality graphs without using additional software. The graph that was previously created using the Annotate facility can now be created using the Graph Template Language (GTL) and the SGRENDER procedure. Here is the program to create the entire graph:

Note: The program below contains numbered annotations that correspond to a discussion below the program. So, if you copy and paste this program into SAS, be sure to delete the annotations before running this code.

data ratio;
  input Age $1-8 type $ Female Male;
datalines;
over 79  A 19 18
70-79    B 15 14
60-69    C 13 12
50-59    A 24 46
40-49    B 26 18
30-39    C 92 61
20-29    A 77 88
under 20 B 42 100
;
run;
 
proc template; 
   define statgraph population; ❶
   begingraph /border=false❷ datacolors=(green blue red); ❸
        entrytitle 'Population Tree' / textattrs=(size=15);❹
   entrytitle 'Distribution of Population by Sex';
   layout lattice ❺/ columns=2 ❻ columnweights=(.55 .45); ❼
      layout overlay ❽/ walldisplay=none y2axisopts=(reverse=true
                         tickvaluehalign=center 
                         display=(tickvalues))  
                         xaxisopts=(displaysecondary=(label) 
                         display=(tickvalues line) reverse=true 
                         labelattrs=(weight=bold));
         barchart category=age response=Female / group=type orient=horizontal 
                                                 yaxis=y2 barlabel=true;
      endlayout;
      layout overlay ❽/ walldisplay=none 
                         yaxisopts=(reverse=true display=none)
                         xaxisopts=(displaysecondary=(label)   
                         display=(tickvalues line)
                         labelattrs=(weight=bold)); 
         barchart category=age response=Male / group=type orient=horizontal 
                                               barlabel=true ;
      endlayout;
   endlayout;
   endgraph;
   end;
 
proc sgrender data=ratio template=population;
run;

As you can see, this program is much simpler than the one created using Version 6 above. Let’s take a closer look at the code.

❶ The TEMPLATE procedure creates a GTL definition called POPULATION with the DEFINE statement.

❷ In the BEGINGRAPH statement, the BORDER=FALSE option turns off the outside border.

❸ The DATACOLORS option defines the colors for the groups.

❹ The ENTRYTITLE statements define the titles for the graph.

❺ A LAYOUT LATTICE block serves as a wrapper for the LAYOUT OVERLAY statements.

❻ The COLUMNS option in the LAYOUT LATTICE block defines the layout of the cells.

❼ Because the graph in the left cell contains the bar values, the COLUMNWEIGHTS option allocates more room for this graph. The values for COLUMNWEIGHTS need to add up to 1.

❽ Two LAYOUT OVERLAY statements define the two cells in this graph.

LAYOUT OVERLAY Code

The two LAYOUT OVERLAY blocks are similar, so we will look closer at only the first one:

layout overlay / walldisplay=none ❶ 
                 y2axisopts=(reverse=true 
                 tickvaluehalign=center display=(tickvalues)) ❷ 
                 xaxisopts=(displaysecondary=(label)display=(tickvalues line)reverse=true ❺ 
                 labelattrs=(weight=bold));
   barchart category=age response=Female ❻/ group=type ❼
                           orient=horizontal ❽ yaxis=y2 ❾ barlabel=true; ❿
endlayout;

❶ The WALLDISPLAY=NONE option turns off the border around the graph.

❷ The REVERSE=TRUE option reverses the Y axis order, TICKVALUEHALIGN=CENTER centers the tick mark values, and the DISPLAY=TICKVALUES option displays only the tick mark values. These options are specified within Y2AXISOPTS.

❸ DISPLAYSECONDARY=(LABEL) displays the X axis label on the X2axis, at the top of the graph.

❹ On the X axis, at the bottom of the graph, the DISPLAY=(TICKVALUES LINE) option displays the tick mark values and the axis line.

❺ The REVERSE=TRUE option also reverses the X axis.

❻ The bar chart contains a bar for each Age. The length of the bars is based on the values of the variable Female with the CATEGORY=AGE and RESPONSE=FEMALE options in the BARCHART statement respectively.

❼ The group variable, TYPE, determines the color of the bars.

❽ The ORIENT=HORIZONTAL option in the BARCHART statement specifies that the bars are horizontal.

❾ YAXIS=Y2 specifies that the values are plotted against the right Y axis, Y2 axis.

❿ The BARLABEL=TRUE option provides labels for the bars.

Here is the graph that is created when you submit this program:

sasgraph-9-4-capabilities02

As this example shows, SAS graphing capabilities have improved over the years just as transportation options have progressed. Be sure to take advantage of these improvements to create helpful visualizations of your data! If you would like to create a similar graph with PROC SGPLOT, refer to Sanjay Matange’s blog post “Butterfly plots.”


Version 6 program

    /* set the graphics environment */
 goptions reset=global gunit=pct border
          ftext=swissb htitle=6 htext=3 dev=png;
 %annomac;
 
    /* create the Annotate data set, POPTREE */
 data poptree;
       /* length and type specification */
    %dclanno;
 
       /* set length of text variable   */
    length text $ 16;
 
 
       /* window percentage for x and y */
    %system(5, 5, 3);
 
       /* draw female axis lines */
    %move(5, 10);
    %draw(40, 10, red, 1, .5);
    %draw(40, 95, red, 1, .5);
 
       /* draw male axis lines */
    %move(56.1, 95);
    %draw(56.1, 10, red, 1, .5);
    %draw(95, 10, red, 1, .5);
 
       /* label categories */
    %label(75.0, 97.0, 'Male', green, 0, 0, 4, swissb, 5);
 
       /* at top */
    %label(25.0, 97.0, 'Female', green, 0, 0, 4, swissb, 5);
    %label(5.0, 5, '100', blue, 0, 0, 4, swissb, 5);
    %label(22.5, 5, ' 50', blue, 0, 0, 4, swissb, 5);
    %label(40.0, 5, ' 00', blue, 0, 0, 4, swissb, 5);
    %label(95.0, 5, '100', blue, 0, 0, 4, swissb, 5);
    %label(75.0, 5, ' 50', blue, 0, 0, 4, swissb, 5);
    %label(56.0, 5, ' 00', blue, 0, 0, 4, swissb, 5);
 
       /* label age */
    %label(48.0, 15.25, 'under 20', blue, 0, 0, 4, swissb, 5);
    %label(48.0, 25.0, '20 - 29', blue, 0, 0, 4, swissb, 5);
    %label(48.0, 36.7, '30 - 39', blue, 0, 0, 4, swissb, 5);
    %label(48.0, 47.4, '40 - 49', blue, 0, 0, 4, swissb, 5);
    %label(48.0, 57.8, '50 - 59', blue, 0, 0, 4, swissb, 5);
    %label(48.0, 68.6, '60 - 69', blue, 0, 0, 4, swissb, 5);
    %label(48.0, 79.3, '70 - 79', blue, 0, 0, 4, swissb, 5);
    %label(48.0, 90.0, 'over 79', blue, 0, 0, 4, swissb, 5);
 
       /* male bars on right */
    %bar(56.2, 10.5, 95.0, 20.0, blue, 0, solid);
    %bar(56.2, 20.71, 90.0, 30.71, green, 0, solid);
    %bar(56.2, 31.42, 80.0, 41.52, red, 0, solid);
    %bar(56.2, 42.14, 62.0, 52.14, blue, 0, solid);
    %bar(56.2, 52.85, 72.0, 62.85, green, 0, solid);
    %bar(56.2, 63.57, 60.0, 73.57, red, 0, solid);
    %bar(56.2, 74.28, 61.0, 84.28, blue, 0, solid);
    %bar(56.2, 85.0, 63.0, 95.0, green, 0, solid);
 
       /* label male bars on right */
    %label(95.0, 20.0, '100', black, 0, 0, 4, swissb, 7);
    %label(90.0, 30.71, '88', black, 0, 0, 4, swissb, 7);
    %label(80.0, 41.52, '61', black, 0, 0, 4, swissb, 7);
    %label(62.0, 52.14, '18', black, 0, 0, 4, swissb, 7);
    %label(72.0, 62.85, '46', black, 0, 0, 4, swissb, 7);
    %label(60.0, 73.57, '12', black, 0, 0, 4, swissb, 7);
    %label(61.0, 84.28, '14', black, 0, 0, 4, swissb, 7);
    %label(62.0, 95.0, '18', black, 0, 0, 4, swissb, 7);
 
       /* female bars on left */
    %bar(39.8, 10.5, 25.0, 20.0, blue, 0, solid);
    %bar(39.8, 20.71, 15.0, 30.7, green, 0, solid);
    %bar(39.8, 31.42, 10.0, 41.42, red, 0, solid);
    %bar(39.8, 42.14, 32.0, 52.14, blue, 0, solid);
    %bar(39.8, 52.85, 33.0, 62.85, green, 0, solid);
    %bar(39.8, 63.57, 36.0, 73.57, red, 0, solid);
    %bar(39.8, 74.28, 35.0, 84.28, blue, 0, solid);
    %bar(39.8, 85.0, 33.0, 95.0, green, 0, solid);
 
       /* label female bars on left */
    %label(25.0, 20.0, '42', black, 0, 0, 4, swissb, 9);
    %label(15.0, 30.7, '77', black, 0, 0, 4, swissb, 9);
    %label(10.0, 41.42, '92', black, 0, 0, 4, swissb, 9);
    %label(32.0, 52.14, '26', black, 0, 0, 4, swissb, 9);
    %label(33.0, 62.85, '24', black, 0, 0, 4, swissb, 9);
    %label(36.0, 73.57, '13', black, 0, 0, 4, swissb, 9);
    %label(35.0, 84.28, '15', black, 0, 0, 4, swissb, 9);
    %label(33.0, 95.0, '19', black, 0, 0, 4, swissb, 9);
 run;
 
    /* define the titles */
 title1 'Population Tree';
 title2 h=4 'Distribution of Population by Sex';
 
    /* generate annotated slide */
 proc gslide annotate=poptree;
 run;
 quit;
tags: Problem Solvers, SAS 9.4, SAS Programmers

Comparing SAS/GRAPH® 9.4 capabilities with SAS/GRAPH® Version 6 was published on SAS Users.

12
Jan

Five great analytics resolutions for 2017

analytics resolutionsThe holiday season is over – and you survived. You’ve made a lot of personal resolutions for 2017 - go to the gym, eat less sugar, save more money, visit Grandma more often. These are all great personal resolutions for 2017, but what about your analytics resolutions? If you are having trouble with your analytics resolutions then let us help you out. The recent release of SAS 9.4 M4 will help you make 2017 your best analytics year yet.

Resolution 1: Build more accurate models faster!

Now you will be able to leverage the power of the two most advanced analytics platforms on the market, SAS 9 and SAS Viya from one interface. Using SAS/Connect, users can call powerful SAS Viya analytics from within a process flow in Enterprise Miner. Would you prefer to use the super-fast, autotuned gradient boosting in SAS Viya? No problem! Call SAS Viya analytics directly from Enterprise Miner using the SAS Viya Code node. Then, from the same process flow you can also call open source models, all from one interface, SAS Enterprise Miner. Do you prefer to use SAS Studio on SAS 9? You will also be able to call SAS Viya analytics from SAS Studio as well. With SAS 9 M4, SAS gives you the ability to use both of SAS’ powerful platforms from one interface.

Resolution 2: Score your unstructured models in Hadoop without moving your data!

Got Hadoop? Got a lot of unstructured data? Now SAS Contextual Analysis allows you to score models in Hadoop using the SAS Code Accelerator add-on. Identify new insights with your unstructured text without ever having to move your data. Score it all in Hadoop. Uncover new trends and topics buried in documents, emails, social media and other unstructured text that is stored in Hadoop. You will be able to do it faster because you won’t have to move that data outside of Hadoop. SAS just keeps getting better in 2017.

Resolution 3: Make better forecasts using the weather!

Through SAS/ETS, econometricians and others wanting to incorporate weather data into their models can now do so directly through two new interface engines. SASERAIN enables SAS users to retrieve weather data from the World Weather Online website. And SASENOAA provides access to severe weather data from the National Oceanic and Atmospheric Administration (NOAA) Severe Weather Data Inventory (SWDI) web service. So now you’ll know why there was that big sales spike for rock salt and snow shovels in July! Who says there is no climate change in 2017?

Resolution 4: Estimate causal effects more efficiently!

The new CAUSALTRT procedure in SAS/STAT estimates the average causal effect of a binary treatment variable T on a continuous or discrete outcome Y. Depending on the application, the variable T can represent an intervention (such as smoking cessation – which is a great 2017 resolution - versus control), an exposure to a condition (such as attending private versus public schools), or an existing characteristic of subjects (such as high versus low socioeconomic status). The CAUSALTRT procedure estimates two types of causal effects: the average treatment effect and the average treatment effect for the treated. And best of all, the causal inference methods that the CAUSALTRT procedure implements are designed primarily for use with data from nonrandomized trials or observational studies, where you observe T and Y without assigning subjects randomly to the treatment conditions.

Resolution 5: Design better factory floors!

A factory floor can be a complicated place, with raw materials coming in one side, and finished products going out the other. Options are virtually unlimited for the placement of materials and equipment – and a poorly designed layout can dramatically reduce production capability. Yet experimenting with different layouts would be extremely costly and time consuming. Thankfully, SAS Simulation Studio (a component of SAS/OR) provides a rich – and animated – environment for testing alternatives and coming up with the most appropriate design. And it can handle any kind of discrete-event simulation, integrating with JMP for experimental design and input analysis, and with JMP and SAS for source data and analysis of simulation results. How will your factory floor simulation impact your productivity in 2017?

tags: analytics, SAS 9.4, SAS Viya

Five great analytics resolutions for 2017 was published on SAS Users.

21
Dec

Easier Space Management for EV Data Mart Tables in 9.4M4

The report-ready SAS Environment Manager Data Mart has been an invaluable addition to SAS 9.4 for SAS administrators. The data mart tables are created and maintained by the SAS Environment Manager Service Architecture Framework and provide a source of data for out-of-the box reports as well as custom reports that any SAS administrator can easily create. As you can imagine, the size of the tables in the data mart can grow quite large over time so balancing the desired time span of reporting and the size of the tables on disk requires some thought. The good news: SAS 9.4 M4 has made that job even easier.

The Environment Manager Data Mart (EVDM) has always provided a configuration setting to determine how many days of resource records to keep in the data mart tables. You can see below that in a fresh SAS 9.4 M4 installation, the default setting for “Number of Days of Resource Records in Data Mart” is set to 60 days. This means that EVDM data records older than 60 days are deleted from tables whenever the data mart ETL process executes.

EV Data Mart Tables in 9.4M4

The space required to house the Environment Manager Data Mart is split across three primary areas.

  • The ACM library tables contain system level information
  • The APM library tables contain audit and performance data culled from SAS logs
  • The KITS library tables contains miscellaneous tables created by data mart kits that collect specialty information about HTTP access, SAS data set access, and such.

Prior to SAS 9.4M4, the ACM and APM libraries duly archived data according to the “Number of Days of Resource Records in Data Mart” setting, but the KITS library did not. For most of the KITS tables this is not such a big deal but for some deployments, the HTTPACCESS table in the KITS library can grow quite large. For administrators who have enabled the VA feed for the Service Architecture Framework, the size of the HTTPACCESS table directly impacts the time it takes to autoload the results of each refresh of the data mart, as well as the amount of memory consumed by the LASR Server used for the Environment Manager Data Mart LASR library.

So what is the big change for SAS 9.4 M4?

The KITS library now respects the “Number of Days of Resource Records in Data Mart” setting and removes data older than the threshold.  If you are a SAS administrator, you can now forget about having to separately manage the KITS library which should simplify space management.

SAS administrators may need to adjust the “Number of Days of Resource Records in Data Mart” setting to strike a balance between the date range requirements for reporting and the amount of disk space they have available for storing the EVDM tables.  With SAS 9.4 M4, however, administrators can rest assured that all EVDM tables will self-manage according to their wishes.

More on the Service Architecture Framework.

tags: SAS 9.4, SAS Administrators, SAS Environment Manager, SAS Professional Services

Easier Space Management for EV Data Mart Tables in 9.4M4 was published on SAS Users.

4
Mar

Add files to a ZIP archive with FILENAME ZIP

In previous articles, I've shared tips about how you can work with SAS and ZIP files without requiring an external tool like WinZip, gzip, or 7-Zip. I've covered:

But a customer approached me the other day with one scenario I missed: how to add SAS data sets to an existing ZIP file. It's a variation of a tip that I've already shared, but with two differences. First, in order to add a data set to a ZIP file, you have to know its physical filename -- not just the LIBNAME.MEMBER reference that you use in SAS procedure steps. And second, I had not shown how to add a new file to an existing ZIP archive -- though it turns out that's pretty simple.

Find the file name for a SAS data set

There are several ways to do this. For my approach, I used the output from PROC CONTENTS. Notice that I had to capture the ODS output (not the OUT= data set) to grab the file name. I wrapped it in a macro for easy reuse. And since I ultimately need a SAS fileref to map to the path, I've assigned one (data_fn) in my macro.

/* macro to assign a fileref to a SAS data set in a Base library */
%macro assignFilerefToDataset(_dataset_name);
    %local outDsName;
    ods output EngineHost=File;
    proc contents data=&_dataset_name.;
    run;
    proc sql noprint;
        select cValue1 into: outDsName 
            from work.file where Label1="Filename";
    quit;
    filename data_fn "&outDsName.";
%mend;

How to add a new member to a ZIP file

Now that I have the source file, I need to designate a destination file in a ZIP archive. The FILENAME ZIP method will create a new ZIP file if one does not yet exist, or it can add to an existing ZIP. To ensure I'm starting from scratch, I assign a simple fileref to my target destination and then delete the file.

/* Assign the fileref - basic file method */
filename projzip "&projectDir./project.zip";
/* Start with a clean slate - delete ZIP if it exists */
data _null_;
    rc=fdelete('projzip');
run;

To create a new ZIP file and designate a path and file name within it, I used the FILENAME ZIP method with the MEMBER= option. Note that I specified the "data/" subfolder in the MEMBER= value; this will place the file into a named subfolder within the archive.

/* Use FILENAME ZIP to add a new member -- CLASS */
/* Put it in the data subfolder */
filename addfile zip "&projectDir./project.zip" 
    member='data/class.sas7bdat';

Then finally, I need to actually "copy" the file into the archive. I do this by streaming the source file into the target fileref byte-by-byte:

/* byte-by-byte copy */
/* "copies" the new file into the ZIP archive */
data _null_;
    infile data_fn recfm=n;
    file addfile recfm=n;
    input byte $char1. @;
    put  byte $char1. @;
run;
 
filename addfile clear;

That's it! I now have a ZIP file with one member entry. Now I can "press repeat" to add a second entry:

%assignFilerefToDataset(sashelp.cars);
/* Use FILENAME ZIP to add a new member -- CARS */
/* Put it in the data subfolder */
filename addfile zip "&projectDir./project.zip" 
    member='data/cars.sas7bdat';
/* byte-by-byte copy */
/* "copies" the new file into the ZIP archive */
data _null_;
    infile data_fn recfm=n;
    file addfile recfm=n;
    input byte $char1. @;
    put  byte $char1. @;
run;
 
filename addfile clear;

Optional: Report on the ZIP file contents

If I want to report on the total contents of the ZIP file now, here's a DATA step and PROC CONTENTS step that does the job:

/* OPTIONAL for reporting */
/* Report on the contents of the ZIP file */
/* Assign a fileref wth the ZIP method */
filename inzip zip "&projectDir./project.zip";
/* Read the "members" (files) from the ZIP file */
data contents(keep=memname);
    length memname $200;
    fid=dopen("inzip");
    if fid=0 then
        stop;
    memcount=dnum(fid);
    do i=1 to memcount;
        memname=dread(fid,i);
        output;
    end;
    rc=dclose(fid);
run;
/* create a report of the ZIP contents */
title "Files in the ZIP file";
proc print data=contents noobs N;
run;

Result:

Files in the ZIP file 

memname
---------------------
data/class.sas7bdat
data/cars.sas7bdat 
N = 2

I hope that this helps to make the FILENAME ZIP method more useful to those who want to try it out. I'm sure that there will be more scenarios that people will ask about; someday, if I write enough blog posts, I'll have it all covered!

Sample program: You can view/download the entire SAS program (containing the snippets I've featured and more) from my GitHub profile.

tags: FILENAME ZIP, SAS 9.4, SAS programming, ZIP files

The post Add files to a ZIP archive with FILENAME ZIP appeared first on The SAS Dummy.

28
Sep

Using SAS DS2 to parse JSON

Thanks to the proliferation of cloud services and REST-based APIs, SAS users have been making use of PROC HTTP calls (to query these web services) and some creative DATA step or PROC GROOVY code to process the JSON results. Such methods get the job done (JSON is simply text, after all), but they aren't as robust as an official JSON parser. JSON is simple: it's a series of name-value pairs that represent an object in JavaScript. But these pairs can be nested within one another, so in order to parse the result you need to know about the object structure. A parser helps with the process, but you still need to know the semantics of any JSON response.

SAS 9.4 introduced PROC JSON, which allows you to create JSON output from a data set. But it wasn't until SAS 9.4 Maintenance 3 that we have a built-in method to parse JSON content. This method was added as a DS2 package: the JSON package.

I created an example of the method working -- using an API that powers our SAS Support Communities! The example queries communities.sas.com for the most recent posts to the SAS Programming category. Here's a small excerpt of the JSON response.

 "post_time": "2015-09-28T16:29:05+00:00",
  "views": {
  "count": 1
  },
  "subject": "Re: How to code for the consecutive values",
  "author": {
  "href": "/users/id/13884",
  "login": "ballardw"

Notice that some items, such as post_time, are simple one-level values. But other items, such as views or author, require a deeper dive to retrieve the value of interest ("count" for views, and "login" for author). The DS2 JSON parser can help you to navigate to those values without you needing to know how many braces or colons or commas are in your way.

Here is an example of the result: a series plot from PROC SGPLOT and a one-way frequency analysis from PROC FREQ. The program also produces a detailed listing of the messages, the topic content, and the datetime stamp.

series

boardfreq
This is my first real DS2 program, so I'm open to feedback. I already know of a couple of improvements I should make, but I want to share it now as I think it's good enough to help others who are looking to do something similar.

The program requires SAS 9.4 Maintenance 3. It also works fine in the most recent version of SAS University Edition (using SAS Studio 3.4). All of the code runs using just Base SAS procedures.

/* DS2 program that uses a REST-based API */
/* Uses http package for API calls       */
/* and the JSON package (new in 9.4m3)   */
/* to parse the result.                  */
proc ds2; 
  data messages (overwrite=yes);
    /* Global package references */
    dcl package json j();
 
    /* Keeping these variables for output */
    dcl double post_date having format datetime20.;
    dcl int views;
    dcl nvarchar(128) subject author board;
 
    /* these are temp variables */
    dcl varchar(65534) character set utf8 response;
    dcl int rc;
    drop response rc;
 
    method parseMessages();
      dcl int tokenType parseFlags;
      dcl nvarchar(128) token;
      rc=0;
      * iterate over all message entries;
      do while (rc=0);
        j.getNextToken( rc, token, tokenType, parseFlags);
 
        * subject line;
        if (token eq 'subject') then
          do;
            j.getNextToken( rc, token, tokenType, parseFlags);
            subject=token;
          end;
 
        * board URL, nested in an href label;
        if (token eq 'board') then
          do;
            do while (token ne 'href');
               j.getNextToken( rc, token, tokenType, parseFlags );
            end;
            j.getNextToken( rc, token, tokenType, parseFlags );
            board=token;
          end;
 
        * number of views (int), nested in a count label ;
        if (token eq 'views') then
          do;
            do while (token ne 'count');
               j.getNextToken( rc, token, tokenType, parseFlags );
            end;
            j.getNextToken( rc, token, tokenType, parseFlags );
            views=inputn(token,'5.');
          end;
 
        * date-time of message (input/convert to SAS date) ;
        * format from API: 2015-09-28T10:16:01+00:00 ;
        if (token eq 'post_time') then
          do;
            j.getNextToken( rc, token, tokenType, parseFlags );
            post_date=inputn(token,'anydtdtm26.');
          end;
 
        * user name of author, nested in a login label;
        if (token eq 'author') then
          do; 
            do while (token ne 'login');
               j.getNextToken( rc, token, tokenType, parseFlags );
            end;
            * get the author login (username) value;
            j.getNextToken( rc, token, tokenType, parseFlags );
            author=token;
            output;
          end;
      end;
      return;
    end;
 
    method init();
      dcl package http webQuery();
      dcl int rc tokenType parseFlags;
      dcl nvarchar(128) token;
      dcl integer i rc;
 
      /* create a GET call to the API                                         */
      /* 'sas_programming' covers all SAS programming topics from communities */
      webQuery.createGetMethod(
         'http://communities.sas.com/kntur85557/' || 
         'restapi/vc/categories/id/sas_programming/posts/recent' ||
         '?restapi.response_format=json' ||
         '&restapi.response_style=-types,-null&page_size=100');
      /* execute the GET */
      webQuery.executeMethod();
      /* retrieve the response body as a string */
      webQuery.getResponseBodyAsString(response, rc);
      rc = j.createParser( response );
      do while (rc = 0);
        j.getNextToken( rc, token, tokenType, parseFlags);
        if (token = 'message') then
          parseMessages();
      end;
    end;
 
  method term();
    rc = j.destroyParser();
  end;
 
  enddata;
run;
quit;
 
/* Add some basic reporting */
proc freq data=messages noprint;
    format post_date datetime11.;
    table post_date / out=message_times;
run;
 
ods graphics / width=2000 height=600;
title '100 recent message contributions in SAS Programming';
title2 'Time in GMT';
proc sgplot data=message_times;
    series x=post_date y=count;
    xaxis minor label='Messages';
    yaxis label='Time created' grid;
run;
 
title 'Board frequency for recent 100 messages';
proc freq data=messages order=freq;
    table board;
run;
 
title 'Detailed listing of messages';
proc print data=messages;
run;
 
title;

I also shared this program on the SAS Support Communities as a discussion topic. If you want to contribute to the effort, please leave me a reply with your suggestions and improvements!

tags: DS2, JSON, REST API, SAS 9.4

The post Using SAS DS2 to parse JSON appeared first on The SAS Dummy.

3
Aug

Using Lua within your SAS programs

With apologies to this candy advertisement from the 1980s:

"Hey, you got your Lua in my SAS program."
"You got your SAS code in my Lua program!"

Announcer: "PROC LUA: Two great programming languages that program great together!"

What is Lua? It's an embeddable scripting language that is often used as a way to add user extensions to robust software applications. Lua has been embedded into SAS for some time already, as it's the basis for new ODS destinations like EXCEL and POWERPOINT. But SAS users haven't had a way to access it.

With SAS 9.4 Maintenance 3 (released July 2015), you can now run Lua code in the new LUA procedure. And from within that Lua code, you can exchange data with SAS and call SAS functions and submit SAS statements. (Running SAS within Lua within SAS -- it's just like Inception.)

Paul Tomas, the developer for PROC LUA, presented a demo of the feature and its usefulness in a recent SAS Tech Talk:


 
Paul also wrote a paper for SAS Global Forum 2015: Driving SAS with Lua.

Like many innovations that find their way into customer-facing features, this new item was added to help SAS R&D complete work for a SAS product (specifically, the new version of SAS Forecast Server). But the general technique was so useful that we decided to add it into Base SAS as a way for you to integrate Lua logic.

PROC LUA can be an alternative to the SAS macro language for injecting logical control into your SAS programs. For example, here's a sample program that generates a SAS data set only if the data set doesn't already exist.

proc lua ;
submit; 
 
-- example of logic control within LUA
if not sas.exists("work.sample") then
    print "Creating new WORK.SAMPLE"
	sas.submit [[
	  data work.sample;
	    set sashelp.class;
	  run;
	 ]]
   else print "WORK.SAMPLE already exists"
 end
 
endsubmit;
run;

First run:

NOTE: Lua initialized.
Creating new WORK.SAMPLE
    data work.sample;
      set sashelp.class;
    run;

And subsequent runs:

NOTE: Resuming Lua state from previous PROC LUA invocation.
WORK.SAMPLE already exists
NOTE: PROCEDURE LUA used (Total process time):
      real time           0.00 seconds
      cpu time            0.00 seconds

Unlike other embedded languages (like PROC GROOVY), Lua runs in the same process as SAS -- and not in a separate virtual machine process like a Java VM. This makes it easy to exchange information such as data and macro variables between your SAS and Lua programming structures.

If you have SAS 9.4M3 and have time to play with PROC LUA, let us know what interesting applications you come up with!

tags: Lua, SAS 9.4

The post Using Lua within your SAS programs appeared first on The SAS Dummy.

20
May

ODS PDF destination in SAS 9.4: Come take a look under the hood!

If you have programmed with SAS in the last 15 years, you have probably had a reason to share your SAS results in PDF format. The ODS PDF destination, much like a well-designed car, has evolved over the years, offering progressively nicer features like security, enhanced image formatting and embedded fonts. I think you’re going to like what SAS 9.4 has to offer!

Let’s discuss three key SAS 9.4 improvements to this beloved, reliable and top rated ODS destination:

  • Pearl default style
  • ODS LAYOUT statement
  • ODSLIST and ODSTEXT procedures

ODS PDF’s new default style: Not just a pretty paint color.

Just like car manufacturers, the ODS developers have improved the “look and feel” of the PDF destination in SAS 9.4 by updating the default style. Styles.Pearl, the new default style for ODS PDF and ODS PRINTER,  is designed with a more modern feel. The headers are no longer shaded, the borders are lighter and we think you’ll agree the results are much more appealing. Here’s a screen shot showing sample PROC PRINT results comparing the new default style, styles.Pearl, with the earlier default, styles.Printer:

Illustration showing differences in SAS 9.4 default ODS Styles.PEARL and previous default Styles.PRINTER

ODS LAYOUT: Production status with robust documentation

The ODS LAYOUT feature, which allows the precise placement of SAS output on a PDF page, was introduced in SAS 9.2. However, it held so much power and required so many feature improvements, it was released in preproduction status. Since that time the SAS programming community has offered feedback on its performance and features, leading to the SAS 9.4 release where the destination is offered with two distinct models: GRIDDED and ABSOLUTE, both of which offer robust “owner’s manuals”.

We think the ABSOLUTE model will be the most popular. You can use it to place tables and images side by side, stack them top to bottom and even overlay output in your PDF files, and works to create output placed on a single PDF page.  The GRIDDED model is more useful when using BY-group data and/or creating multi page output.

The ABSOLUTE LAYOUT syntax uses a pair of ODS LAYOUT statements (ODS LAYOUT START and ODS LAYOUT END) to define the area on the page to which you will write ODS REGIONs.  ODS REGION statements define the location and size of regions to which you will write tables, text or images. Both statements use the X= argument to define the horizontal “start” position, and Y= to define the vertical start position of your output. If you do not use either of these, the output starts in the upper left corner (0,0). Height= and Width= arguments can be used on both statements to control the size of the layouts and regions.

Here is sample code for defining a sleek layout page within your ODS PDF destination:

  • Create a PDF file with FILE=, turn off the bookmark/table of contents with NOTOC, and with NOGTITLE make all Graph titles part of the PDF file, instead of embedding them in the Graph images.
    ods pdf file="file.pdf" notoc nogtitle ;
  • Define a LAYOUT that is 7.25 inches high and 10.5 inches wide.  The keyword ABSOLUTE is not necessary as it is implied if neither the COLUMNS= nor the ROWS= option is used. Previously we have set the system option ORIENTATION= to LANDSCAPE on an OPTIONS statement and set a Title and Footnote, whose text will be placed outside the layout.
    ods layout start height=7.25in width=10.5in ;
  • Define the first REGION. The region is placed inside my layout, starting .75 inch from my top margin (y=.75in) and one inch from my left margin (x=1in). This region is allocated 3.5inches of space across my page (width=3.5in).
    ods region x=1in y=.75in width=3.5in ;

The ODS LAYOUT and REGION statements used above are part of Sample 55808.  We’ll use the full code to generate the following output from SAS 9.4, and I’ll explain how to assemble all the pieces using other new features in SAS 9.4:

SAS 9.4 ODS output using ODS LAYOUT statement

PROC ODSTEXT: easily insert paragraphs of text

The ODSTEXT procedure helps us format paragraph text that would previously have been written with DATA _null_ / FILE PRINT logic or ODS <destination> TEXT= syntax. However, neither of these methods allow for elegant style control and line spacing, nor do either of those strategies integrate well with the table of contents.  Now using PROC ODSTEXT, we can format paragraphs of text (or just single words if preferred!), with ease.

Here’s the first PROC ODSTEXT code used in our screen shot shown above:

  
proc odstext;
    p "The ODSTEXT procedure offers a powerful tool for inserting paragraph style text into your ODS destinations. "  / style=systemtitle;
    p  "The ODSTEXT procedure allows us to: format text with style control offered with the statement option style=,  use data sets, and add bullets."  / style=systemtitle;
    p '  ';
    p "A close sibling is PROC ODSLIST, used to generate the following bulleted list of improved ODS PDF features:" /  style=header;
run;

Here are the results:

PROC ODSTEXT output showing three sizes and

PROC ODSLIST: easily create bulleted lists

Closely related to the ODSTEXT procedure is PROC ODSLIST, which allows us to create nicely formatted, indented and bulleted lists from a SAS dataset. Here’s an example:

 proc odslist data=features;
   item feature /style={bullet="disc"};
   end;
 run;

Here are the results:

PROC ODSLIST output
There’s an added bonus in SAS 9.4: the ODSLIST and ODSTEXT procedures can be used in ANY of the non-LISTING ODS destinations.  Try them out in ODS POWERPOINT and ODS RTF too!

Adding the bar chart to our layout

The image in our PDF file is created with the PROC SGPLOT and highlights a feature new in SAS 9.4, the SYMBOLCHAR statement.  The data and program are described in Sample 54315 and a recent Problem Solvers blog. The key to sizing and inserting the output in the upper right corner of our PDF file is specifying the following ODS REGION statement and the ODS GRAPHICS statement height= option before the PROC SGPLOT code.

ods region x=4in y=.5in ; 
ods graphics on / reset noborder height=3.5in;

Adding the table to the PDF

The bottom table is created by PROC REPORT along with some “power assist” from PROCs ODSTEST and FORMAT. This region is defined only with a Y= specification as I want REPORT’s table to take up the width of the entire region. The REPORT table will be centered by default, but ODSTEXT= will not, hence the j=c style override.

ods region y=4in; 

proc format;
value myfmt low-40="^{style [foreground=red] ^{unicode 2193} "
           40-high="^{style [foreground=green] ^{unicode 2191} ";
run;

proc odstext ;
   p " ";
   p "A subset of SASHELP.CARS where MPG_HIGHWAY is greater than 40 " / style={font=(", Albany",10pt,bold) just=c};
 run;

proc report data=sashelp.cars(where=(mpg_highway gt 40 and cylinders eq 4)) spanrows
style(report)={posttext="^{style [font_weight=bold] PROC FORMAT uses inline formatting and the Unicode style function to differentiate models based on the combined MPG}" };
col make model mpg_highway mpg_city avg type enginesize horsepower drivetrain;
define make / order;
define avg  / computed "Combined MPG" format=8. style(column)={posttext=myfmt.};
compute avg;
   avg=sum(mpg_highway.sum,mpg_city.sum)/2;
endcomp;
run;

Here are the results:

Using ODS LAYOUT feature to create and place a PROC REPORT table
We “put the brakes on” by closing the LAYOUT and the PDF destination:

ods layout end;
ods pdf close;

I hope you have enjoyed “kicking the tires” on my favorite destination. Are you excited about getting this new and updated model? If so, give your SAS dealership (ahem, representative) a call and upgrade today!

Please read the fine print:

If you have been an intrepid ODS LAYOUT coder prior to SAS 9.4, you are likely to see different results from your ODS LAYOUT/REGION statements when you move to SAS 9.4. If using the legacy style, styles.printer, does not correct the results, use Scott Huntley’s paper “An Insider’s Guide to ODS LAYOUT Using SAS® 9.4” . It is an excellent resource for coders moving ODS LAYOUT code from previous releases.

If you are moving from a SAS release prior to version 9.2, Scott and I wrote this paper “Getting the Right Report (Again): Your Compatibility Guide for ODS PDF 9.2” to discuss changes that took place for ODS PDF in that version.

Scott and Woody Middleton introduce you to the new features for ODS PDF in SAS 9.3 in "A Different Point of View with ODS PDF in SAS®9.3".

SAS has provided printable tip sheets for ODS PDF, ODS LAYOUT, PROC ODSTEXT, PROC ODSLIST, ODS GRAPHICS and more.

tags: ods, Problem Solvers, reporting, SAS 9.4, SAS Programmers

The post ODS PDF destination in SAS 9.4: Come take a look under the hood! appeared first on SAS Users.

20
May

Using LIBNAME XLSX to read and write Excel files

When you weren't watching, SAS did it again. We smuggled Yet Another Excel Engine into a SAS release.

SAS 9.2 Maintenance 2 added the XLSX engine, which allows you to read and write Microsoft Excel files as if they were data sets in a library. The big advantage of using this engine is that it accesses the XLSX file directly, and doesn't use the Microsoft data APIs as a go-between. (LIBNAME EXCEL and LIBNAME PCFILES rely on those Microsoft components.) That means that you can use this engine on Windows or Unix systems without having to worry about bitness (32-bit versus 64-bit) or setting up a separate PC Files Server process.

The XLSX engine does require a license for SAS/ACCESS to PC Files. Are you a SAS University Edition user? The SAS/ACCESS product is part of that package, so this technique works there. It's an easy way to get well-formed Excel data into your SAS process.

/* because Excel field names often have spaces */
options validvarname=any;
 
libname xl XLSX '/folders/myfolders/sas_tech_talks_15.xlsx';
 
/* discover member (DATA) names */
proc datasets lib=xl; quit;
 
libname xl CLEAR;

Example output:

xl_contents
Once the library is assigned, I can read the contents of a spreadsheet into a new SAS data set:

/* because Excel field names often have spaces */
options validvarname=any;
 
libname xl XLSX '/folders/myfolders/sas_tech_talks_15.xlsx';
 
/* read in one of the tables */
data confirmed;
  set xl.confirmed;
run;
 
libname xl CLEAR;

And here's the result in my SAS University Edition:

xl_confirmed
Sometimes you need just one value from a spreadsheet. That's a common use case for dynamic data exchange (DDE), which isn't as feasible as it once was. You can use FIRSTOBS and OBS options to control how much data you retain:

/* read in just one value */
data _null_;
  set xl.confirmed (firstobs=6 obs=6 keep='Job title'n);
  call symput('VALUE','Job Title'n);
run;
%put &value;

Output:

 76         %put &value;
 Testing Manager,  Quality-driven User Experience Testing

You can also use the XLSX engine to create and update XLSX files.

libname xlout XLSX '/folders/myfolders/samples.xlsx';
 
data xlout.cars;
  set sashelp.cars;
run;
 
data xlout.classfit;
  set sashelp.classfit;
run;
 
data xlout.baseball;
  set sashelp.baseball;
run;
 
data xlout.air;
  set sashelp.air;
run;
 
libname xlout clear;

Here is my output in Microsoft Excel with all of these data sets now as sheets:

xl_xlsxout
Remember, you can also create Microsoft Excel files with Base SAS by using ODS EXCEL -- experimental in 9.4 Maintenance 2 but production in Maintenance 3 (coming soon).

The XLSX libname is different from the EXCEL and PCFILES engines in other ways. For example, the XLSX engine does not support Excel named ranges (which can surface a portion of a spreadsheet as a discrete table). Also, you won't see the familiar "$" decoration around the spreadsheet names when they are surfaced in the library within SAS. If you need that sort of flexibility, you can use PROC IMPORT to provide more control over exactly what Excel content is brought into SAS and how.

One other IMPORTANT caution: The XLSX engine is a sequential access engine in that it processes data one record after the other. The engine starts at the beginning of the file and continues in sequence to the end of the file. Some techniques to MODIFY the data in-place will not work. Also, some SAS data viewers cannot render the data from the XLSX engine. SAS VIEWTABLE and SAS Enterprise Guide and even SAS Studio can't open these tables directly in the data grid view. VIEWTABLE gives you a nice message, but SAS Enterprise Guide simply "hangs" in the attempt. For that reason, I recommend using DATA step to copy the Excel content that you want to another SAS library, then CLEAR the XLSX library to avoid accidentally opening a table in a viewer that won't support it. (This is currently a bug in SAS Enterprise Guide that should be fixed in a future release.)

I have found LIBNAME XLSX to be a quick, convenient method to bring in Excel data on any SAS platform. If you have SAS 9.4 Maintenance 2 or later, try it out! Let me know how it works for you by sharing a comment here.

tags: excel, ODS EXCEL, SAS 9.4, SAS University Edition, xlsx

The post Using LIBNAME XLSX to read and write Excel files appeared first on The SAS Dummy.

Back to Top