5
Dec

What fuel is 1000 times dirtier than diesel?

I recently read an interesting article about petroleum coke (petcoke). A lot of it is produced in the US, and lately a lot of it is consumed (burned) in India ... contributing to air pollution there. The article mentioned some numbers in the text, but the data was really begging to [...]

The post What fuel is 1000 times dirtier than diesel? appeared first on SAS Learning Post.

Read More
5
Dec

DOSUBL() - execute macro code in a data step

The DOSUBL() function, new in SAS 9.4, requires us to rethink the way we look at passing values to a SAS macro. Previously I would create a series of macro variables using PROC SQL with the INTO clause such as :macvar1 - :macvar999 or by using CALL SYMPUTX().

If you try to pass data set variables into a macro they are rendered as the column names, not the column values. The way I used to do this required a %DO loop inside a macro. Now I can use the CATS() function to concatenate strings inside the DOSUBL function. As you can see in the below code, I am passing in a data set variable directly into a macro and using the RESOLVE() function to render its value. The return value of DOSUBL is zero if it was successful.


%macro createfile( filename =, text = ) ;
   filename temp "&filename." ;
   data _null_ ;
     file temp ;
     if not missing( "&text." ) then put "&text." ;
   run ;
   filename temp clear ;
%mend ;

%let path = %sysget(HOME) ;
data files ;
input filename $64. ;
datalines ;
&path./a.txt
&path./b.txt
;
run ;

/* Old technique */
%macro sql1() ;
proc sql noprint ;
select filename
into :file1 - :file999
from files ;
quit ;

%do I = 1 %to &sqlobs. ;
%createfile( filename = &&file&i ) ;
%end ;
%mend ;
%sql1()

/*New technique */
data _null_ ;
  set files ;
  rc = dosubl(cats('%createfile(filename=',resolve(filename),')'));
run ;
  Read More
5
Dec

מחקר האוצר על "שכר המינימום ונזקיו" – 13 שנים מאוחר יותר

בשנת 2004 פירסם משרד האוצר מחקר על "שכר המינימום ונזקיו", ובו הזהירו חכמי המשרד מפני הקטסטרופה הכלכלית שתתרחש אם יועלה שכר המינימום. נזכרתי בעובדה הזו בעקבות ציוץ מתבדח בטוויטר . המחקר ההוא היה מופת של סטטיסטיקה רעה: "מדגם" מוטה בכוונה תחילה – נבחרו רק הנתונים שהתאימו לאג'נדה, בלבול בין מתאם לסיבתיות, שימוש במדדים בעייתיים, זריקת […] Read More
4
Dec

Little things go a long way

In my previous post, I described a new options to control the widths of the caps for Whiskers, Error and Limit bars.  This topic could have been titled "Little things go a long way", as such details really make for a good graph. In a similar manner, another detail issue [...]

The post Little things go a long way appeared first on Graphically Speaking.

Read More
4
Dec

How to scrape data from a web page using SAS

The internet is rich with data, and much of that data seems to exist only on web pages, which -- for some crazy reason -- are designed for humans to read. When students/researchers want to apply data science techniques to analyze collect and analyze that data, they often turn to [...]

The post How to scrape data from a web page using SAS appeared first on The SAS Dummy.

Read More
4
Dec

Controlling your formats

During my 35 years of using SAS® software, I have found the CNTLIN and CNTLOUT options in the FORMAT procedure to be among the most useful features that I routinely suggest to other SAS users. The CNTLIN option enables you to create user-defined formats from a SAS data set (input [...]

Controlling your formats was published on SAS Users.

Read More
4
Dec

חמש דרכים לתקן את הסטטיסטיקה

ב-28.11.2017 הופיע בכתב העת Nature מאמר שנשא את הכותרת הפרובוקטיבית Five ways to fix statistics. המאמר נכתב לאור "משבר השחזוריות" (reproducibility crisis) בו חשים היום חוקרים מתחומים רבים, כלומר הקושי ההולך וגובר לשחזר תוצאות מחקריות במחקר נוסף בלתי תלוי. יש הטוענים כי אחת הסיבות למשבר הזה הוא שימוש לא נכון בסטטיסטיקה. עורכי Nature פנו לחמישה […] Read More
4
Dec

Mean imputation in SAS

Imputing missing data is the act of replacing missing data by nonmissing values. Mean imputation replaces missing data in a numerical variable by the mean value of the nonmissing values. This article shows how to perform mean imputation in SAS. It also presents three statistical drawbacks of mean imputation. How [...]

The post Mean imputation in SAS appeared first on The DO Loop.

Read More
2
Dec

Connect to Server macro

In my last post I created an encoded password file. This macro utilizes the password to make a connection to a SAS/Connect server.

After your password is read in from the password file, you need to specify the name of the SAS/Connect server and its listening port. Once complete, you specify a few options, all of which are described in the below code. After that, you invoke the SIGNON command. The rest of the code does some fundamental error handling.

It is always a good idea to encapsulate code into a single reusable module for maintainability. In this case that container is a SAS macro. What follows is the entire source code to connect to a SAS/Connect server.


/**************************************************************************
* Program: connecttoserver.sas
* Author: Tom Bellmer
* Created: 20171202
* SAS Version: SAS 9.4 (TS1M3)
* OS: Linux
* Purpose: Establish SAS/Connect session to 1SG
* Usage: %connecttoserver()
* Note: Be sure to add SIGNOFF SASGRID ; when finished.
**************************************************************************/

%macro connecttoserver() ;
%global sasgrid ;

/* read in the encoded password */
%inc "%sysget(HOME)/drowssap.dwp" ;

/* 7551 is the listening port of the Connect Spawner */
%let sasgrid = yourhostname 7551 ;

options
comamid = tcp /* Communications access method */
remote = sasgrid /* Points to &sasgrid, no & required */
netencryptalgorithm = aes /* value of remote server */
noconnectmetaconnection /* Bypass metadata authentication */
;

signon sasgrid
cmacvar = signonstatus /* macro var to validate connection */
connectstatus = no /* No Status window */
noscript /* Not using script, this is faster */
signonwait = yes /* wait for connection */
username = "&sysuserid."
password = "&password."
;

%if &signonstatus. = 0 %then %do ;
%put %str(N)OTE: Signon to SASGRID on host &syshostname. is successful. ;
%end ;

%else %if &signonstatus. = 2 %then %do ;
%put %str(N)OTE: Connected to existing session on host &syshostname.. ;
%end ;

%else %do ;
%put %str(E)RROR: Signon to SASGRID failed, (return code: &signonstatus.) ;
%abort cancel ;
%end ;
%mend ;

/*EOF: connecttoserver.sas */
Read More
2
Dec

Encoded Password

I was using SAS/Connect to submit jobs across SAS grids. This all worked for me when I hardcoded my password but not for others because they have different passwords.

The solution I came up with was to utilize PROC PWENCODE with the sas003 method. If your organization does not have SAS/SECURE then you will need to change to the default method of sas002. The sas003 method uses a 256-bit key plus 16-bit salt to encode passwords and will also be FIPS 140-2 compliant. FIPS 140-2 is a U.S. government computer security standard used to approve cryptographic modules.

Every UNIX user has a HOME environmental variable which contains the user's specific HOME directory that can be obtained via the SYSGET(HOME) command. Note that HOME must be in uppercase.

In order to obfuscate the password filename, I used reversed "password" to "drowssap" with an extension of "dwp" or "pwd" backwards. Once the file is created on UNIX/Linux, I used SYSTASK to set the permission to 400 or read only by the creator of the file.

Once the file has been created, it is simply a matter of run the following line of code:


%INCLUDE “%sysget(HOME)/drowssap.dpw” ;

Here is an example of the generated output file followed by the full source code.


/*Password created: 02DEC2017:15:18:29*/
%global password ;
%let password = {SAS003}24F08AC76DBA3044B14EE06846D015D1D250 ;


/**************************************************************************
* Program: setpassword.sas
* Author: Tom Bellmer
* Created: 20171202
* SAS Version: SAS 9.4 (TS1M3)
* OS: LIN X64
* Purpose: Create a proc pwencode(d) password and set the permissions
* to read only or 400
* Usage: %setpassword( password = )
* Notes: After created %inc "%sysget(HOME)/drowssap.dwp" ;
**************************************************************************/

%macro setpassword( password = ) ;
%local pwfile ;

%if %isblank( &password. ) %then %do ;
%put %str(E)RROR: Must pass a non empty password value. ;
%return ;
%end ;

filename junk temp ;
proc pwencode
in = "&password."
out = junk /* outputs to &_pwencode. */
method = sas003 /* 256-bit fixed key plus a random salt value */
;
run ;
filename junk clear ;

/* drowssap is pasword backwards and dwp is pwd (password) in reverse */
%let pwfile = %sysget(HOME)/drowssap.dwp ;

/* delete old version of the file if it exists */
%if %sysfunc( fileexist( "&pwfile." ) ) %then %do ;
systask command "rm &pwfile." wait ;
%end ;

filename pwd "&pwfile." new ;
data _null_ ;
file pwd ;
put "/*Password created: %sysfunc(strip(%sysfunc(datetime(),datetime20.)))*/"
/ '%global password ;'
/ '%let password = ' "&_pwencode. ;"
;
run ;
filename pwd clear ;

/* set permissions to read for user only. */
systask command "chmod 400 &pwfile." wait ;
%mend ;

/*EOF: setpassword.sas */

Read More
Back to Top