`9Sc@sddlZddlZddlZddlZddlTddlTddlmZddlmZddlm Z ddl Tddl Tddl Tddl TyddlTWnek reZnXeZdgZdefdYZdfd YZd fd YZdS( iN(t*(tgui(ttxt(tutilstPhotometerDatabaseWidgetcBseZdZeZdZddZdZdZ dZ dZ dZ dZ d Zd Zd Zd Zd ZdZdZdZedZdZdZdZdZerdZndZdZdZ dZ!dZ"dZ#RS(slWidget responsible for loading of Sun photometer data files for the aerosol profile retrieval algorithm.s data/AERONETcCsTtj||tj|j|_d|_d|_t j j j |_ d|_ g|_ t}td}|j|t|_|jjtj|jjj|j|j|j|j|jtd}|j|t|_|jjj|j|j|j|j|jtd}|j|t|_|jjj|j|j|j|j|jtjd}t|d|_|jj|jj j|j!|j|j|j"ttjdd|_#|j#j$t%|j#j&d |j#j j|j'|j|j#ttjd d|_(|j(j&d |j(j j|j)|j|j(ttjd d|_*|j*j&d |j*j j|j+|j|j*|j,dddd|j-|t.r%|jj/n|j0t%|j1|j2|j3dS(Ns1.5sAeronet &site:s&Year:s Data l&evel:tworlds &Update datasmicrosoft-office-exceltsNExport the currently selected photometer data file into Microsoft Excel formattreloadsCSave changes and reload the currently selected photometer data filet parameterss.Select root folder for the photometer databasei(4tStatusSignalingWidgett__init__Rt absolutePathtdefaultDatabaseDirtdatabaseRootDirtNonet siteInfoListtsiteNametdatetimetnowtyeart dataLeveltdataListt QHBoxLayouttQLabelt addWidgett QComboBoxtsiteNameComboBoxtsetSizeAdjustPolicytAdjustToContentstcurrentIndexChangedtconnecttonComboBoxIndexChangedtsetBuddyt yearComboBoxtdataLevelComboBoxRtloadIcont QPushButtontupdateDataButtonthidetclickedtonUpdateDataButtonClickedt addStretchtexcelExportButtont setEnabledtFalset setToolTiptonExcelExportButtonClickedt reloadButtontonReloadButtonClickedt openButtontonOpenButtonClickedtsetContentsMarginst setLayouttaeronetDownloaderPluginLoadedtshowtpopulateSiteNameComboBoxtpopulateYearComboBoxtpopulateDataLevelComboBoxtsyncComboBoxIndices(tselftparenttlayoutt siteNameLabelt yearLabeltdataLevelLabelt worldIcon((s code\PhotometerDatabaseWidget.pyR :sv                           cCs|jS(N(R(R;((s code\PhotometerDatabaseWidget.pyt getSiteNamescCs|jS(N(R(R;((s code\PhotometerDatabaseWidget.pytgetYearscCs|jS(N(R(R;((s code\PhotometerDatabaseWidget.pyt getDataLevelscCs|jS(s Return the list of 'PhotometerInput' instances loaded from the currently selected photometer data file. If no data have been loaded, an empty list is returned. Note: Currently, the data list returned by this function may be directly edited by the caller.(R(R;((s code\PhotometerDatabaseWidget.pyt getDataLists cCs|jdks|jdkr"dSxtrytj|j|jPWq%tj k r}t j |d|j t j t jBt j }|t jkrPqq%Xq%WdS(s/Save changes that have been made by the user to 'aot675Corr' attributes of the currently loaded list of 'PhotometerInput' instances in the appropriate AOT correction data file. Note: this function is called automatically whenever current photometer data file is changed.Ns"Failed to save AOT correction file(RRRtTruetPhotometerInputtupdateAotCorrFileForDataListtgetAotCorrFilePathRRtErrort QMessageBoxtcriticalttexttRetrytIgnore(R;tetanswer((s code\PhotometerDatabaseWidget.pytsaveAotCorrFiles   cCs|tj|dZ|jdtj|j|jd|j|jd|j|jd|jWdQX|j dS(sSave attributes of the currently selected photometer data file in the given 'QSettings' instance, and also save any changes that the user might have introduced to 'aot675Corr' attributes of the currently loaded list of 'PhotometerInput' instances.RR RRRN( RtSettingsGroupertsetValueRt portablePathR RRRRR(R;tsettings((s code\PhotometerDatabaseWidget.pyt saveSettingss c CsItj|d|jdj|_|jdkrI|j|_ntj|j|_y=t|jdj|_ |j dkrd |_ nWnt k rd |_ nX|jdj \|_ }|sd |_ n|jdj}|dkrd|_n d|_Wd QX|jt|j|jd S( sLoad attributes of the photometer data file stored in the given 'QSettings' instance, load the AERONET site list file, and load the photometer data file itself if it is available and its site name is present in the AERONET site list.RR RRRRs2.0s1.5N(RRStvaluettoStringR R RR tstrRRtUnicodeEncodeErrorttoIntRRR7RFR:tloadPhotometerData(R;RVtconversionResultR((s code\PhotometerDatabaseWidget.pyt loadSettingss*       cCs|jdks$t|jdkr(dSt||}d}|jt|jdj|jdj}xgtdt|jD]M}|jt|j|j|j|j}||kr|}|}qqW|j|jS(sReturn name of the AERONET site that is the nearest one with respect to the given geodetic latitude and longitude (in degrees), or 'None' if AERONET site list file has not been loaded yet or is not available.iiN( RRtlent GeodeticPointt distToPointtlatitudet longitudetrangeR(R;RcRdt targetPointt nearestIndextnearestDistancetitdist((s code\PhotometerDatabaseWidget.pytgetNearestSiteNames$ # #  cCs\||_||_|j|j|jdkrXtj|ddtj|ndS(sjLoad photometer data file for the given site name and year, and the currently selected data level.sAERONET data not availables*AERONET data are not available for year %sN( RRR:R]RRKRLRt quoteNumber(R;RR((s code\PhotometerDatabaseWidget.pytsetSiteNameAndYear1s     cCs/tjj|j|jt|jd|jS(suReturn path to the directory where files related to the currently selected photometer data subset are stored.sLevel (tostpathtjoinR RRZRR(R;((s code\PhotometerDatabaseWidget.pytgetAeronetFileDirFscCs=d|jd|jd|jf}tjj|j|S(sReturn path to the original Dubovik data file that corresponds to the currently selected photometer (AERLID) data file.s%02d0101_%02d1231_%s.dubovikid(RRRnRoRpRq(R;tfileName((s code\PhotometerDatabaseWidget.pytgetDubovikFilePathMs!cCs=d|jd|jd|jf}tjj|j|S(s|Return path to the currently selected photometer data file (a Dubovik AERONET data file processed with AERLID tool).s%02d0101_%02d1231_%s.lidar.txtid(RRRnRoRpRq(R;Rr((s code\PhotometerDatabaseWidget.pytgetAerlidFilePathVs!cCs=d|jd|jd|jf}tjj|j|S(s}Return path to the AOT correction data file that corresponds to the currently selected photometer (AERLID) data file.s%02d0101_%02d1231_%s.aot.txtid(RRRnRoRpRq(R;Rr((s code\PhotometerDatabaseWidget.pyRI_s!cCsdS(s/Return path to the AERONET site list data file.s'code/data/AERONET/aeronet_locations.txt((R;((s code\PhotometerDatabaseWidget.pytgetSiteListFilePathhsc Csxtj|jP|jjd|_trt|jjj }|jj t ||tj d}|jj |dn|s|jjtdSytj|j|_Wn7tjk r}|jjt|j|jdSX|jjt|jjddx:tt|jD]#}|jj||j|jq7WWdQX|jjdS(sFill in the combo box meant for selection the AERONET site name. The combo box will contain the site list itself, and also an item that will make it possible to download the most recent version of the site list file and reload it. If 'loadSiteInfoList' if 'False', only the extra utility item is added to the combobox. Otherwise, the AERONET site list file is loaded, and a corresponding error message is set on failure. The simpler initialization is used in the constructor, and the right one is used in 'loadSettings', thus making sure that 'errorMessageChanged' signal could be received by the parent widget in case of an error.RsUpdate site listNtcmpcSst|j|jS(N(RvR(txty((s code\PhotometerDatabaseWidget.pyts( Rt SignalBlockerRtclearRRR5t QFontMetricstfonttheightt setIconSizetQSizeR#taddItemR1R+R,tAeronetSiteListLoadertloadFileRuRRJtsetErrorMessageRMRFtsortReR`t insertItemRR=tactivate(R;tloadSiteInfoListt iconHeightRARPRi((s code\PhotometerDatabaseWidget.pyR7ps6       cCsptj|jX|jjtjjj}x1td|dD]}|jjt |qFWWdQXdS(sQFill in the combo box meant for selection of year of the photometer data.iiN( RRzR!R{RRRReRRZ(R;t currentYearR((s code\PhotometerDatabaseWidget.pyR8s  cCsJtj|j2|jj|jjd|jjdWdQXdS(sKFill in the combo box meant for selection of photometer data level.s1.5s2.0N(RRzR"R{R(R;((s code\PhotometerDatabaseWidget.pyR9s c Csd}|jdk rj|jdk rjxCtt|jD])}|j|j|jkr:|}Pq:q:Wn|dkrd|_ntj|j|jj|WdQX|j j t |j }|dkrd|_ ntj|j |j j|WdQX|j j |j}|dks.ttj|j |j j|WdQX|jj|jdk o{|j dk dS(sSynchronize currently selected items of the combo boxes with actual attributes of the photometer data file stored in the local variables. If any of the values stored in the local variables is not present in the respective combo box, that value is reset to 'None', and current item of the combo box is cleared by setting its current index to -1. Note: this function does not load the currently selected photometer data file; use 'loadPhotometerData' for that.iN(RRRReR`RRzRtsetCurrentIndexR!tfindTextRZRR"RtAssertionErrorR%R+(R;t siteIndexRit yearIndextdataLevelIndex((s code\PhotometerDatabaseWidget.pyR:s*       c Cs |j2|jjt|jj|jdkr7dS|jdksU|j dkr|j j|jdk oy|j dk |j ddS|j }|j }tjj| rtjj| r|j dtj|dStjj|rtjj| sWtjj|rntjj|rntjj|tjj|krntjd|jdS|j}tjj|sd}ny6|jdtj|ttj|||_Wn'tjk r}|j |jdSX|j|jjt|jjdS(s[Load the currently selected photometer data file as defined by the local variables.Ns&Photometer data subset is not selecteds+AERONET database file (%s) is not availableis%Loading photometer input file (%s)...(RR*R+R,tdataListChangedtemitRRRRR%RRsRtRnRotisfileRt quotePathtgetmtimetQTimert singleShott runAerlidToolRItsetProgressMessageRFRGtreadDataListFromAerlidRJRMtclearStatusMessage(R;t dubovikPatht aerlidPatht aotCorrPathRP((s code\PhotometerDatabaseWidget.pyR]sF     & %$$    cCs~t||jrp|jt|j}|j|jdkrz||_|jd|_|jqzn |jdS(sDownload the most recent version of the AERONET site list file, update contents of the site name combo box appropriately if the download succeeds, and then restore the currently selected site name in the combo box.N( tdownloadAeronetSiteListFileRuR7RFRR:RRRR](R;tpreviousSiteName((s code\PhotometerDatabaseWidget.pytupdateSiteListFileMs      cCs|jdtj|jtt|dd|j|j}|j|jdk rz|j |jdS|j dS(sRun AERLID tool for the original Dubovik data file that corresponds to the currently selected AERLID file, and then reload the updated AERLID data file.sProcessing AERONET data (%s)...scode/binary/AERLIDg0v@g@g@N(g0v@g@g@( RRRRsRFtAerlidToolDialogRttexec_tgetErrorMessageRRR](R;taerlidToolDialog((s code\PhotometerDatabaseWidget.pyRzs   cCsN|j|jj}trg||jjdkrg|jj|dksVt|jdSn|dkrt|jj||_ n d|_ |j j}|dkrt |j j||_ n d|_ |jj}|dkstt|jj||_|jj|j dk o<|j dk |jdS(sUUpdate local variables that define the currently selected photometer data file and load data from the newly selected file. If 'Update site list' item has been selected in the AERONET site name combo box, download the most recent version of the site list file and update the combo box' contents appropriately.isUpdate site listNi(RRRt currentIndexR5tcounttitemTextRRRZRRR!tintRR"RR%R+R](R;tindexRRR((s code\PhotometerDatabaseWidget.pyRs*       cCsAtr=t||j|j|j|jr=|jq=ndS(sDownload the most recent version of the Dubovik AERONET data file corresponding to the currently selected photometer data subset, process the downloaded file with AERLID tool and then reload the updated AERLID data file.N(R5tdownloadAeronetDubovikDataFileRRRRsR(R;((s code\PhotometerDatabaseWidget.pyR(s cCstj|j}tj|d|d}|jr=dStj|}|jdt ytj ||j Wn,tj k r}t j|d|jnX|jdS(Ns7Export photometer data fine into Microsoft Excel formatsMicrosoft Excel files (*.xls)s=Exporting photometer data file into Microsoft Excel format...sFailed to save the file(RGtgetExcelExportFilePathRst QFileDialogtgetSaveFileNametisNullRR RRFtexportDataListToExcelRRJRKRLRMR(R;tdefaultExcelFilePatht excelFilePathRP((s code\PhotometerDatabaseWidget.pyR.s     cCs/|j|jt|j|jdS(sSave changes made to 'aot675Corr' attributes of the currently loaded list of 'PhotometerInput' instances, reload the AERONET site list file and then reload the currently selected photometer data file.N(RRR7RFR:R](R;((s code\PhotometerDatabaseWidget.pyR0s   c Cs<d}tjj|js$d}n |j}tj|||}|jsZ|dkr^dStj|}|j dk st g|j D]}|j ^q}yag}xTtj |D]C}tjjtjj||r||kr|j|qqWWn2tk r6tj||dtj|dSXt|dkr|jt|dkrdtj|d}nt|d krd tj|dtj|df}nt|d kr d tj|dtj|dtj|d f}n:d tj|dtj|dtj|d f}t|dkrnd}d} d} nd}d} d} t|} | j|| jdtj|||| | f| jd| jtj| jtjtjB| jtjkrdSn|j tj||_|j!dS(s@Ask the user to select a new value for the database root folder.s.Select root folder for the photometer databaset.RNs&Selected folder (%s) is not accessibleiis%sis %s and %sis%s, %s, and %ss%s, %s, %s, etc.s a subfoldertnameu doesn’tt subfolderstnamesudon’ts[Selected folder (%s) contains %s (%s), whose %s %s coincide with any of AERONET site names.sIDo you really want this folder to be the root of the photometer database?("RnRotisdirR RtgetExistingDirectoryRRR RRRRtlistdirRptappendtOSErrorRKRLRR`RtquotetsetWindowTitletsetTexttsetInformativeTexttsetIcontWarningtsetStandardButtonstYestNoRRRR]( R;t dialogCaptiontdirPathtinfot siteNamest rootSubDirsRt nameListStrt subfoldersStrtnamesStrtdontStrtmsgBox((s code\PhotometerDatabaseWidget.pyR2st    !    *       N($t__name__t __module__t__doc__t pyqtSignalRR RR RBRCRDRERRRWR_RkRmRqRsRtRIRuR,R7R8R9R:R]R5RRRR(R.R0R2(((s code\PhotometerDatabaseWidget.pyR.s>  h    "  -     B  5 K -  5   tAeronetSiteInfocBseZdZRS(sDescription of an AERONET site stored in the AERONET site list file. Attributes: - 'siteName': string holding the AERONET site name. - 'longitude', 'latitude': geodetic coordinates of the site, in degrees. - 'altitude': site's elevation, in meters.(RRR(((s code\PhotometerDatabaseWidget.pyRhsRcBsDeZdZdZdZdZdZdZdZRS(s=Utility class used for parsing of the AERONET site list file.cCsad|_|jjd|_|jjd|_|jjd|_|jjd|_dS(sAInitialize auxiliary attributes used to describe the data format.t Site_NamesLongitude(decimal_degrees)sLatitude(decimal_degrees)sElevation(meters)N(RsLongitude(decimal_degrees)sLatitude(decimal_degrees)sElevation(meters)(theadersRt nameIndextlongitudeIndext latitudeIndext altitudeIndex(R;((s code\PhotometerDatabaseWidget.pyR }s  cCstjj|s1tjdtj|nyt||_Wn-tk rstjdtj|nXdtj||_ zBy|j SWn-tk rtjdtj|nXWd|jj XdS(sLoad the specified AERONET site list file. Return a list of 'AeronetSiteInfo' instances. Raise 'txt.Error' if loading fails or file format is invalid.s,Aeronet site list file (%s) is not availables9Aeronet site list file (%s) may not be opened for readings/Aeronet site list file format is invalid (%s): s*Failed to read Aeronet site list file (%s)N( RnRoRRRJRtopentfiletIOErrort errorPrefixtreadFiletclose(R;tfilePath((s code\PhotometerDatabaseWidget.pyRs"      cCsg}|jj|jjjjd}x-tt|jD]}|j||qGWd|_x|jD]}|jjd}t }|j ||_ |j ||j |_|j ||j|_|j ||j|_|j||jd7_qtW|S(s4Load the opened data file referenced in 'self.file'.t,ii(RtreadlinetstriptsplitReR`RttestColumnHeadert currentLineRt readSiteNameRtreadFloatValueRRdRRcRtaltitudeR(R;Rt dataHeadersRitlinet dataStringstsiteInfo((s code\PhotometerDatabaseWidget.pyRs&       cCs|t|krRtj|jdtj|dtj|j|fn|||j|krtj|jdtj|dtj||tj|j|fndS(sCheck if the given data column header has the expected value. Raise 'txt.Error' on failure. 'headerList' should be the comlete list of header strings read from the file. 'i' should be the index of the data column.sdata column %s (%s) in missingis'header of column %s is %s instead of %sN(R`RRJRRlRR(R;t headerListRi((s code\PhotometerDatabaseWidget.pyRs 1  cCs|t|krRtj|jdtj|dtj|jdfnyt||SWnNtk rtj|jdtj|dtj|jdfnXdS(s#Parse a floating-point value read from the data file. Return the parsed value. Raise 'txt.Error' on failure. 'dataList' should be the comlete list of data strings read from a single line of the data file. 'i' should be the index of the data column.s#missing data in column %s (line %s)is%invalid number in column %s (line %s)N(R`RRJRRlRtfloatt ValueError(R;RRi((s code\PhotometerDatabaseWidget.pyRs  !  cCs|jt|krBtj|jdtj|jdn||j}tjd|dkrtj|jdtj|jdtj |fn|S(s Parse and check the site name read from the AERONET site list file. Return a string holding the site name. Raise 'txt.Error' on failure. 'dataList' should be the comlete list of data strings read from a single line of the data file.smissing site name in line %sis[a-zA-Z0-9_\-]+$s!invalid site name in line %s (%s)N( RR`RRJRRlRtretmatchRR(R;RR((s code\PhotometerDatabaseWidget.pyRs   -( RRRR RRRRR(((s code\PhotometerDatabaseWidget.pyRrs ! *  (RRntos.pathRt PyQt4.QtCoret PyQt4.QtGuit common.utilsRRRtcommon.GeodeticPointtcommon.StatusSignalingWidgetRRGtAeronetDownloadert ImportErrorR,R5RFt__all__R RRR(((s code\PhotometerDatabaseWidget.pyts0             =