variance-covariance output

questions concerning analysis/theory using program MARK

variance-covariance output

Postby Jaume » Wed Mar 29, 2006 6:09 am

Hello,
I am trying to obtain the variance-covariance of a multi-state model with a relatively large number of "real parameters" (470). Two of the three output options in MARK, write to NOTEPAD and Copy to Clipboard, produce error messages on lack of memory and crash the program (the memory of the system I am using is ca. 1500 Mb.). The third option, writing up the matrix in a dbf file works out fine, but I do not have Dbase to open it. When I import the dbf file into Excel the file is trimmed because Excel only allows for 253 columns (and this model has more than 400). In trying to import this file into Acces, I find that Acces does not recognise the file. Are there any good tips on how to read a dbf file without Dbase?

Thanks a lot in advance.
Jaume Forcada
Jaume
 

Re: variance-covariance output

Postby cooch » Wed Mar 29, 2006 12:25 pm

Jaume wrote:Hello,
I am trying to obtain the variance-covariance of a multi-state model with a relatively large number of "real parameters" (470). Two of the three output options in MARK, write to NOTEPAD and Copy to Clipboard, produce error messages on lack of memory and crash the program (the memory of the system I am using is ca. 1500 Mb.).


Well, that actually isn't a lot of memory - I have colleagues working with *large* multi-state data sets that have their computers loaded up for 4 GB of RAM, or better.

But, more to the point - first approach to solving the problem is in recognizing that the standard notepad in Windows is pretty useless. My generally recommendation is to simply replace it with something 'real'. This is easy to do. All you need to do is (i) locate notepad.exe on your machine (on a 'typical' XP machine, notepad.exe is found in in several places, so you may need to fool around a bit to figure out which one MARK uses), (ii) rename notepad.exe to notepad.exe.original (or some such), then (iii) rename the executable for the free editor you install to notepad.exe. EditPad Lite (available at http://www.editpadpro.com) works quite well for this. Once youve made this switch, now any call to notepad will start up the new, better editor.

Once you have something better than Notepad, try to open your file again - this *trick* might (and often will) solve your problem. Failing that...

The third option, writing up the matrix in a dbf file works out fine, but I do not have Dbase to open it. When I import the dbf file into Excel the file is trimmed because Excel only allows for 253 columns (and this model has more than 400). In trying to import this file into Acces, I find that Acces does not recognise the file. Are there any good tips on how to read a dbf file without Dbase?


Well, I use SAS for this purpose, but you can do it fairly easily in [R], which is free. Of course, this means installing and learning to use [R], but sooner or later, you'll need *real* tools (like SAS, or S-Plus, or [R]), not Excel, so you might as well decide sooner, rather than later, to invest some time in learning to use one of the many that are available. If you use the [R] approach, you'll need to install the RODBC package, which lets [R] handle ODBC format data files (Excel, dBase, etc...).
cooch
 
Posts: 1654
Joined: Thu May 15, 2003 4:11 pm
Location: Cornell University

Postby Jaume » Thu Mar 30, 2006 3:24 am

Hello EGC

sooner or later, you'll need *real* tools (like SAS, or S-Plus, or [R]), not Excel, so you might as well decide sooner, rather than later, to invest some time in learning to use one of the many that are available.


I am a regular user/programmer of/in S and I solved the problem using S-Plus before your reply, but thank you very much anyway. S-Plus for Windows worked after I tried R. Apparently R for Windows does not have the function read.dbf which seems to be available in R for other platforms. For some reason, the connecting functions in the R package RODBC did not recognise the DBF file from MARK. I will delve into that. I also tried MATLAB, which I use regularly as well, but it did not import dbf files. (I abandoned SAS long time ago to avoid paying too many expensive software licenses.)

Well, that actually isn't a lot of memory - I have colleagues working with *large* multi-state data sets that have their computers loaded up for 4 GB of RAM, or better.


Thanks again. Now that we start a new fiscal year I will ask again our budget holders for more RAM.
Jaume
 

Postby cooch » Thu Mar 30, 2006 12:36 pm

I am a regular user/programmer of/in S...


...but thought you'd use Excel...for fun? ;-)


Glad you got it figured out.
cooch
 
Posts: 1654
Joined: Thu May 15, 2003 4:11 pm
Location: Cornell University

Postby cooch » Sat Apr 01, 2006 1:43 pm

Jaume wrote: For some reason, the connecting functions in the R package RODBC did not recognise the DBF file from MARK. I will delve into that. I also tried MATLAB, which I use regularly as well, but it did not import dbf files.


My mistake - RODBC works fine under Linux. Under Windows, it seems to be 'broken', so use this instead:

Code: Select all
library(foreign)
cov <- read.dbf("RealVCMatrix.dbf")


cov contains everything you're after (or should).
cooch
 
Posts: 1654
Joined: Thu May 15, 2003 4:11 pm
Location: Cornell University


Return to analysis help

Who is online

Users browsing this forum: No registered users and 4 guests