{"id":440,"date":"2016-09-30T14:39:43","date_gmt":"2016-09-30T21:39:43","guid":{"rendered":"https:\/\/bionmr.mbi.ucla.edu\/?page_id=440"},"modified":"2016-10-11T14:56:58","modified_gmt":"2016-10-11T21:56:58","slug":"processing-with-hmsistnmrpipe","status":"publish","type":"page","link":"https:\/\/bionmr.mbi.ucla.edu\/?page_id=440","title":{"rendered":"Processing with hmsIST\/nmrPipe"},"content":{"rendered":"<p><span style=\"text-decoration: underline;\"><strong>Data conversion for data acquired with TS3<\/strong><\/span><\/p>\n<p>Getting the correct bruk2pipe conversion script can be tricky. The <span style=\"color: #800000;\">convert2pipe<\/span> AU program usually doesn&#8217;t give correct results (it will interpret a 3D experiment as a 2D). It&#8217;s better to use the <span style=\"color: #800000;\">bruker<\/span> command from a shell. Here is an example for a 3D NUS dataset that was collected with 152 hypercomplex NUS points. Type <span style=\"color: #800000;\">bruker<\/span> in a shell:<\/p>\n<ul>\n<li>Set the dimension count to: <span style=\"color: #800000;\">3<\/span><\/li>\n<li>Set Digital Oversampling Correction to: <span style=\"color: #800000;\">During Processing<\/span><\/li>\n<li>Set total\/valid points in y dimension to: <span style=\"color: #800000;\">4\/2<\/span><\/li>\n<li>Set total\/valid points in z dimension to: (eg) <span style=\"color: #800000;\">152\/76<\/span><\/li>\n<li>Acquisition Mode: <span style=\"color: #800000;\">x:DQD \u00a0 \u00a0y:Real \u00a0 \u00a0z:Real<\/span><\/li>\n<li>If necessary, manually enter lines to label the y dimension as Rance-Kay, and to ouput the data properly. In this case F2 was Echo-Antiecho, so we label the y dimension as Rance-Kay.<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<p style=\"text-align: center;\"><strong>The conversion script: <span style=\"color: #800000;\">fid.com<\/span><\/strong><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-495\" src=\"https:\/\/bionmr.mbi.ucla.edu\/wp-content\/uploads\/2016\/10\/NUS10-300x119.png\" alt=\"NUS10\" width=\"790\" height=\"313\" srcset=\"https:\/\/bionmr.mbi.ucla.edu\/wp-content\/uploads\/2016\/10\/NUS10-300x119.png 300w, https:\/\/bionmr.mbi.ucla.edu\/wp-content\/uploads\/2016\/10\/NUS10.png 530w\" sizes=\"auto, (max-width: 790px) 100vw, 790px\" \/><\/p>\n<p>&nbsp;<\/p>\n<p><span style=\"text-decoration: underline;\"><strong>Data conversion for data acquired with TS2\/TS1\/xwinnmr<\/strong><\/span><\/p>\n<p>Continuing with the example from <a href=\"https:\/\/bionmr.mbi.ucla.edu\/?page_id=448\">Acquiring NUS data using TopSpin 2, TopSpin 1, or xwinnmr<\/a>:<\/p>\n<p>An HNCACB with 64 (uniform) complex points in F1 (13C) and 55 (uniform) complex points in F2 (15N). 30% sampling gave 1058 hypercomplex NUS points in the sampling schedule. \u00a0Both indirect dimensions were States-TPPI.<\/p>\n<p>&nbsp;<\/p>\n<p>The conversion script can be generated either with the <span style=\"color: #800000;\">convert2pipe<\/span> AU program or with the <span style=\"color: #800000;\">bruker<\/span> program.<\/p>\n<p style=\"text-align: center;\"><strong>The conversion script: <span style=\"color: #800000;\">fid.com<\/span><\/strong><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-499\" src=\"https:\/\/bionmr.mbi.ucla.edu\/wp-content\/uploads\/2016\/10\/NUS_FT10-300x110.png\" alt=\"NUS_FT10\" width=\"900\" height=\"330\" srcset=\"https:\/\/bionmr.mbi.ucla.edu\/wp-content\/uploads\/2016\/10\/NUS_FT10-300x110.png 300w, https:\/\/bionmr.mbi.ucla.edu\/wp-content\/uploads\/2016\/10\/NUS_FT10.png 655w\" sizes=\"auto, (max-width: 900px) 100vw, 900px\" \/><\/p>\n<p>&nbsp;<\/p>\n<p><span style=\"text-decoration: underline;\"><strong>Data processing<\/strong><\/span><\/p>\n<p>After conversion to nmrPipe format, the data processing is the same no matter which way the data was acquired.<\/p>\n<p>The direct dimension is processed normally in nmrPipe. The script used is <span style=\"color: #800000;\">ft1xyz.com<\/span>:<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-501\" src=\"https:\/\/bionmr.mbi.ucla.edu\/wp-content\/uploads\/2016\/10\/NUS_FT11-300x112.png\" alt=\"NUS_FT11\" width=\"916\" height=\"342\" srcset=\"https:\/\/bionmr.mbi.ucla.edu\/wp-content\/uploads\/2016\/10\/NUS_FT11-300x112.png 300w, https:\/\/bionmr.mbi.ucla.edu\/wp-content\/uploads\/2016\/10\/NUS_FT11.png 593w\" sizes=\"auto, (max-width: 916px) 100vw, 916px\" \/><\/p>\n<p>Initially the phase values in ft1xyz.com will be 0. Run the script once, then use nmrDraw to view the first FID, which will be in the file <span style=\"color: #800000;\">test001.nus<\/span> in the directory <span style=\"color: #800000;\">xyz<\/span>. Each nus file has the data from\u00a0<em>one hypercomplex point<\/em>. For 3D data, this will be 4 1D spectra. The first one is the one at the bottom.\u00a0Phase this spectrum interactively. Then go back into the ft1xyz.com script and write in the new phase values. You can run it again to check that it&#8217;s right.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-503\" src=\"https:\/\/bionmr.mbi.ucla.edu\/wp-content\/uploads\/2016\/10\/NUS11-300x139.png\" alt=\"NUS11\" width=\"486\" height=\"225\" srcset=\"https:\/\/bionmr.mbi.ucla.edu\/wp-content\/uploads\/2016\/10\/NUS11-300x139.png 300w, https:\/\/bionmr.mbi.ucla.edu\/wp-content\/uploads\/2016\/10\/NUS11-768x355.png 768w, https:\/\/bionmr.mbi.ucla.edu\/wp-content\/uploads\/2016\/10\/NUS11-1024x474.png 1024w, https:\/\/bionmr.mbi.ucla.edu\/wp-content\/uploads\/2016\/10\/NUS11.png 1863w\" sizes=\"auto, (max-width: 486px) 100vw, 486px\" \/><\/p>\n<p>Next, run the script ft1.com. This script is almost identical to ft1xyz.com. The only difference is the way it writes out the data.<\/p>\n<p style=\"text-align: center;\"><strong>ft1.com<\/strong><\/p>\n<p style=\"text-align: left;\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-505\" src=\"https:\/\/bionmr.mbi.ucla.edu\/wp-content\/uploads\/2016\/10\/NUS_FT12-300x142.png\" alt=\"NUS_FT12\" width=\"913\" height=\"432\" srcset=\"https:\/\/bionmr.mbi.ucla.edu\/wp-content\/uploads\/2016\/10\/NUS_FT12-300x142.png 300w, https:\/\/bionmr.mbi.ucla.edu\/wp-content\/uploads\/2016\/10\/NUS_FT12.png 595w\" sizes=\"auto, (max-width: 913px) 100vw, 913px\" \/><\/p>\n<p style=\"text-align: left;\">Now we&#8217;re ready to reconstruct the missing data. This is done with the program <span style=\"color: #800000;\">istHMS<\/span>.<\/p>\n<p style=\"text-align: left;\">Edit the script <span style=\"color: #800000;\">ist.csh<\/span>:<\/p>\n<p style=\"text-align: center;\"><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-507 alignnone\" src=\"https:\/\/bionmr.mbi.ucla.edu\/wp-content\/uploads\/2016\/10\/NUS_FT14-300x171.png\" alt=\"NUS_FT14\" width=\"414\" height=\"236\" srcset=\"https:\/\/bionmr.mbi.ucla.edu\/wp-content\/uploads\/2016\/10\/NUS_FT14-300x171.png 300w, https:\/\/bionmr.mbi.ucla.edu\/wp-content\/uploads\/2016\/10\/NUS_FT14.png 439w\" sizes=\"auto, (max-width: 414px) 100vw, 414px\" \/><\/p>\n<p style=\"text-align: left;\">In the ist.csh script, the parameters -xN 64 and -yN 128 are the size of the two indirect dimensions after zero filling (x = F2, y = F1). These values can equal the number of (uniform) complex points in the 2 dimensions, or they can be a higher power of 2. In this case, 64 &#8211;&gt; 128 and 55 &#8211;&gt; 64. The only other thing you should change here is <span style=\"color: #800000;\">itr<\/span> &#8211; the number of iterations that the <span style=\"color: #800000;\">istHMS<\/span> program does. It will be faster with fewer iterations, but the quality of the reconstruction may degrade. 400 iterations is usually enough.<\/p>\n<p style=\"text-align: left;\">This script starts the <span style=\"color: #800000;\">istHMS<\/span> program. It uses the <span style=\"color: #800000;\">sched<\/span> file, which is the schedule of NUS points that your experiment ran. Unfortunately, the 2 columns of the sched file are often in the wrong order. All of our 3D NUS experiments are done with <span style=\"color: #800000;\">aqseq 321<\/span>. So F2 is the &#8220;first&#8221; indirect dimension, and F1 is the &#8220;second&#8221; indirect dimension. Therefore, the correct order of the columns in the <span style=\"color: #800000;\">sched<\/span> file is:<\/p>\n<p style=\"text-align: left;\"><span style=\"color: #800000;\">F2 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0F1<\/span><\/p>\n<p style=\"text-align: left;\">In other words, the column on the left should be the vplist (the F2 NUS points), and the column on the right should be the vclist (the F1 NUS points). If they&#8217;re in the wrong order, you must swap the columns. You can delete (or rename) the file <span style=\"color: #800000;\">sched<\/span>. The files <span style=\"color: #800000;\">vclist<\/span> and <span style=\"color: #800000;\">vplist<\/span> are in your data directory. There is a script called <span style=\"color: #800000;\">sched_combine_vp_vc.py<\/span> that will make a new sched file with the vc\/vp lists in the proper order.<\/p>\n<p style=\"text-align: left;\">The run the script <span style=\"color: #800000;\">run.local<span style=\"color: #000000;\">, which starts the script <span style=\"color: #800000;\">ist.csh<\/span> and <span style=\"color: #800000;\">istHMS<\/span>:<\/span><\/span><\/p>\n<p style=\"text-align: left;\">\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0<img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-506\" src=\"https:\/\/bionmr.mbi.ucla.edu\/wp-content\/uploads\/2016\/10\/NUS_FT13-300x14.png\" alt=\"NUS_FT13\" width=\"643\" height=\"30\" srcset=\"https:\/\/bionmr.mbi.ucla.edu\/wp-content\/uploads\/2016\/10\/NUS_FT13-300x14.png 300w, https:\/\/bionmr.mbi.ucla.edu\/wp-content\/uploads\/2016\/10\/NUS_FT13.png 644w\" sizes=\"auto, (max-width: 643px) 100vw, 643px\" \/><\/p>\n<p style=\"text-align: left;\">This does the recontruction and writes the files <span style=\"color: #800000;\">test*.nus<\/span> into the directory <span style=\"color: #800000;\">yzx<\/span>\u00a0 \u00a0 \u00a0(* = 001, 002, etc)<\/p>\n<p style=\"text-align: left;\">Now run the script <span style=\"color: #800000;\">phf2pipe.com<span style=\"color: #000000;\">. This reorders the data into planes (back into the format that nmrPipe expects).\u00a0<\/span><\/span><\/p>\n<p style=\"text-align: center;\"><strong>phf2pipe.com<\/strong><\/p>\n<p style=\"text-align: left;\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-508\" src=\"https:\/\/bionmr.mbi.ucla.edu\/wp-content\/uploads\/2016\/10\/NUS_FT15-300x33.png\" alt=\"NUS_FT15\" width=\"855\" height=\"94\" srcset=\"https:\/\/bionmr.mbi.ucla.edu\/wp-content\/uploads\/2016\/10\/NUS_FT15-300x33.png 300w, https:\/\/bionmr.mbi.ucla.edu\/wp-content\/uploads\/2016\/10\/NUS_FT15.png 766w\" sizes=\"auto, (max-width: 855px) 100vw, 855px\" \/><\/p>\n<p style=\"text-align: left;\">Now that the reconstruction is done, the 2 indirect dimensions can be processed normally by nmrPipe:<\/p>\n<p style=\"text-align: center;\"><strong>An example processing script for F1 and F2: ft23_xyz.com<\/strong><\/p>\n<p style=\"text-align: left;\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-509\" src=\"https:\/\/bionmr.mbi.ucla.edu\/wp-content\/uploads\/2016\/10\/NUS_FT16-300x226.png\" alt=\"NUS_FT16\" width=\"874\" height=\"658\" srcset=\"https:\/\/bionmr.mbi.ucla.edu\/wp-content\/uploads\/2016\/10\/NUS_FT16-300x226.png 300w, https:\/\/bionmr.mbi.ucla.edu\/wp-content\/uploads\/2016\/10\/NUS_FT16.png 538w\" sizes=\"auto, (max-width: 874px) 100vw, 874px\" \/><\/p>\n<p>&nbsp;<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-512 alignright\" src=\"https:\/\/bionmr.mbi.ucla.edu\/wp-content\/uploads\/2016\/10\/NUS1-300x278.png\" alt=\"NUS1\" width=\"618\" height=\"573\" srcset=\"https:\/\/bionmr.mbi.ucla.edu\/wp-content\/uploads\/2016\/10\/NUS1-300x278.png 300w, https:\/\/bionmr.mbi.ucla.edu\/wp-content\/uploads\/2016\/10\/NUS1-768x712.png 768w, https:\/\/bionmr.mbi.ucla.edu\/wp-content\/uploads\/2016\/10\/NUS1.png 954w\" sizes=\"auto, (max-width: 618px) 100vw, 618px\" \/><\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>You can view the data in nmrDraw:<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p><span style=\"text-decoration: underline;\"><strong>Processing scripts<\/strong><\/span><\/p>\n<p>You can get processing scripts from the <a href=\"http:\/\/gwagner.med.harvard.edu\/intranet\/hmsIST\/index.html\">Wagner lab website<\/a>. There are also example processing scripts and executables in the directory\u00a0\/home\/peterson\/NMR\/NUS\/Processing_Scripts:<\/p>\n<p><span style=\"color: #800000;\">fid.com<\/span> \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0(there are different versions of fid.com for different types of experiments)<br \/>\n<span style=\"color: #800000;\">ft1xyz.com<\/span><br \/>\n<span style=\"color: #800000;\">ft1.com<\/span><br \/>\n<span style=\"color: #800000;\">run.local<\/span><br \/>\n<span style=\"color: #800000;\">parallel<\/span><br \/>\n<span style=\"color: #800000;\">sched_combine_vp_vc.py<\/span><br \/>\n<span style=\"color: #800000;\">ist.csh<\/span><br \/>\n<span style=\"color: #800000;\">istHMS<\/span><br \/>\n<span style=\"color: #800000;\">phf2pipe.com<\/span><br \/>\n<span style=\"color: #800000;\">phf2pipe<\/span><br \/>\n<span style=\"color: #800000;\">ft23_xzy.com<\/span><\/p>\n<p><span style=\"color: #000000;\">Remember that you may have to modify some of these &#8211; especially the nmrPipe conversion and processing scripts.<\/span><\/p>\n<p>&nbsp;<\/p>\n<p><a href=\"https:\/\/bionmr.mbi.ucla.edu\/?page_id=408\">back to Non-Uniform Sampling Experiments<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Data conversion for data acquired with TS3 Getting the correct bruk2pipe conversion script can be tricky. The convert2pipe AU program usually doesn&#8217;t give correct results (it will interpret a 3D experiment as a 2D). It&#8217;s better to use the bruker command from a shell. Here is an example for a 3D NUS dataset that was [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-440","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/bionmr.mbi.ucla.edu\/index.php?rest_route=\/wp\/v2\/pages\/440","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/bionmr.mbi.ucla.edu\/index.php?rest_route=\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/bionmr.mbi.ucla.edu\/index.php?rest_route=\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/bionmr.mbi.ucla.edu\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/bionmr.mbi.ucla.edu\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=440"}],"version-history":[{"count":23,"href":"https:\/\/bionmr.mbi.ucla.edu\/index.php?rest_route=\/wp\/v2\/pages\/440\/revisions"}],"predecessor-version":[{"id":522,"href":"https:\/\/bionmr.mbi.ucla.edu\/index.php?rest_route=\/wp\/v2\/pages\/440\/revisions\/522"}],"wp:attachment":[{"href":"https:\/\/bionmr.mbi.ucla.edu\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=440"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}