The GNU Awk User's Guide

29/7/2016
TheGNUAwkUsersGuide
TheGNUAwkUsersGuide
Next:Foreword3,Up:(dir)[Contents][Index]
GeneralIntroduction
Thisfiledocumentsawk,aprogramthatyoucanusetoselectparticularrecordsinafileandperformoperationsupon
them.
Copyright1989,1991,1992,1993,19962005,2007,20092015
FreeSoftwareFoundation,Inc.
ThisisEdition4.1ofGAWK:EffectiveAWKProgramming:AUsersGuideforGNUAwk,forthe4.1.2(orlater)
versionoftheGNUimplementationofAWK.
Permissionisgrantedtocopy,distributeand/ormodifythisdocumentunderthetermsoftheGNUFreeDocumentation
License,Version1.3oranylaterversionpublishedbytheFreeSoftwareFoundationwiththeInvariantSectionsbeing
GNUGeneralPublicLicense,withtheFrontCoverTextsbeingAGNUManual,andwiththeBackCoverTextsas
in(a)below.AcopyofthelicenseisincludedinthesectionentitledGNUFreeDocumentationLicense.
1.TheFSFsBackCoverTextis:YouhavethefreedomtocopyandmodifythisGNUmanual.
Foreword3:
Foreword4:
Preface:
GettingStarted:
InvokingGawk:
Regexp:
ReadingFiles:
Printing:
Expressions:
PatternsandActions:
Arrays:
Functions:
LibraryFunctions:
SamplePrograms:
AdvancedFeatures:
Internationalization:
Debugger:
ArbitraryPrecision
Arithmetic:
DynamicExtensions:
LanguageHistory:
Installation:
Notes:
BasicConcepts:
Glossary:
SomenicewordsaboutthisWebpage.
Morenicewords.
WhatthisWebpageisaboutbriefhistoryandacknowledgments.
Abasicintroductiontousingawk.Howtorunanawkprogram.Commandlinesyntax.
Howtorungawk.
Allaboutmatchingthingsusingregularexpressions.
Howtoreadfilesandmanipulatefields.
Howtoprintusingawk.Describestheprintandprintfstatements.Alsodescribes
redirectionofoutput.
Expressionsarethebasicbuildingblocksofstatements.
Overviewsofpatternsandactions.
Thedescriptionanduseofarrays.Alsoincludesarrayorientedcontrolstatements.
Builtinanduserdefinedfunctions.
ALibraryofawkFunctions.
Manyawkprogramswithcompleteexplanations.
Stuffforadvancedusers,specifictogawk.
Gettinggawktospeakyourlanguage.
Thegawkdebugger.
Arbitraryprecisionarithmeticwithgawk.
Addingnewbuiltinfunctionstogawk.
Theevolutionoftheawklanguage.
Installinggawkundervariousoperatingsystems.
Notesaboutaddingthingstogawkandpossiblefuturework.
Averyquickintroductiontoprogrammingconcepts.
Anexplanationofsomeunfamiliarterms.
https://www.gnu.org/software/gawk/manual/gawk.html
1/479
29/7/2016
TheGNUAwkUsersGuide
Copying:
GNUFreeDocumentation
License:
Index:
Yourrighttocopyanddistributegawk.
ThelicenseforthisWebpage.
History:
Names:
ThisManual:
Conventions:
ManualHistory:
HowToContribute:
Acknowledgments:
Runninggawk:
Oneshot:
ReadTerminal:
Long:
ExecutableScripts:
Comments:
Quoting:
DOSQuoting:
SampleDataFiles:
VerySimple:
TwoRules:
MoreComplex:
Statements/Lines:
OtherFeatures:
When:
IntroSummary:
CommandLine:
Options:
OtherArguments:
NamingStandardInput:
EnvironmentVariables:
AWKPATHVariable:
AWKLIBPATHVariable:
OtherEnvironment
Variables:
ExitStatus:
IncludeFiles:
LoadingSharedLibraries:
Obsolete:
Undocumented:
InvokingSummary:
RegexpUsage:
EscapeSequences:
RegexpOperators:
BracketExpressions:
LeftmostLongest:
ComputedRegexps:
GNURegexpOperators:
Casesensitivity:
Thehistoryofgawkandawk.
Whatnametousetofindawk.
UsingthisWebpage.Includessampleinputfilesthatyoucanuse.
TypographicalConventions.
BriefhistoryoftheGNUprojectandthisWebpage.
Helpingtosavetheworld.
Acknowledgments.
Howtorungawkprogramsincludescommandlinesyntax.
Runningashortthrowawayawkprogram.
Usingnoinputfiles(inputfromthekeyboardinstead).
Puttingpermanentawkprogramsinfiles.
Makingselfcontainedawkprograms.
Addingdocumentationtogawkprograms.
Morediscussionofshellquotingissues.
QuotinginWindowsBatchFiles.
SampledatafilesforuseintheawkprogramsillustratedinthisWebpage.
Averysimpleexample.
Alesssimpleonelineexampleusingtworules.
Amorecomplexexample.
Subdividingorcombiningstatementsintolines.
OtherFeaturesofawk.
Whentousegawkandwhentouseotherthings.
Summaryoftheintroduction.
Howtorunawk.
Commandlineoptionsandtheirmeanings.
Inputfilenamesandvariableassignments.
Howtospecifystandardinputwithotherfiles.
Theenvironmentvariablesgawkuses.
Searchingdirectoriesforawkprograms.
Searchingdirectoriesforawksharedlibraries.
Theenvironmentvariables.
RegexpSummary:
Regularexpressionssummary.
ConceptandVariableIndex.
gawksexitstatus.
Includingotherfilesintoyourprogram.
Loadingsharedlibrariesintoyourprogram.
ObsoleteOptionsand/orfeatures.
UndocumentedOptionsandFeatures.
Invocationsummary.
HowtoUseRegularExpressions.
Howtowritenonprintingcharacters.
RegularExpressionOperators.
Whatcangobetween[...].
Howmuchtextmatches.
UsingDynamicRegexps.
OperatorsspecifictoGNUsoftware.
Howtodocaseinsensitivematching.
2/479
29/7/2016
TheGNUAwkUsersGuide
Records:
awksplitrecords:
gawksplitrecords:
Fields:
NonconstantFields:
ChangingFields:
FieldSeparators:
DefaultFieldSplitting:
RegexpFieldSplitting:
SingleCharacterFields:
CommandLineField
Separator:
FullLineFields:
FieldSplittingSummary:
ConstantSize:
SplittingByContent:
MultipleLine:
Getline:
PlainGetline:
Getline/Variable:
Getline/File:
Getline/Variable/File:
Getline/Pipe:
Getline/Variable/Pipe:
Getline/Coprocess:
Getline/Variable/Coprocess:
GetlineNotes:
GetlineSummary:
ReadTimeout:
Commandlinedirectories:
InputSummary:
InputExercises:
Print:
PrintExamples:
OutputSeparators:
OFMT:
Printf:
BasicPrintf:
ControlLetters:
FormatModifiers:
PrintfExamples:
Redirection:
SpecialFD:
SpecialFiles:
OtherInheritedFiles:
SpecialNetwork:
SpecialCaveats:
CloseFilesAndPipes:
OutputSummary:
Controllinghowdataissplitintorecords.
Howstandardawksplitsrecords.
Howgawksplitsrecords.
Anintroductiontofields.
NonconstantFieldNumbers.
ChangingtheContentsofaField.
Thefieldseparatorandhowtochangeit.
Howfieldsarenormallyseparated.
Usingregexpsasthefieldseparator.
Makingeachcharacteraseparatefield.
SettingFSfromthecommandline.
OutputExercises:
Values:
Exercises.
Constants,Variables,andRegularExpressions.
Makingthefulllinebeasinglefield.
Somefinalpointsandasummarytable.
Readingconstantwidthdata.
DefiningFieldsByContent
Readingmultilinerecords.
Readingfilesunderexplicitprogramcontrolusingthegetlinefunction.
Usinggetlinewithnoarguments.
Usinggetlineintoavariable.
Usinggetlinefromafile.
Usinggetlineintoavariablefromafile.
Usinggetlinefromapipe.
Usinggetlineintoavariablefromapipe.
Usinggetlinefromacoprocess.
Usinggetlineintoavariablefromacoprocess.
Importantthingstoknowaboutgetline.
SummaryofgetlineVariants.
Readinginputwithatimeout.
Whathappensifyouputadirectoryonthecommandline.
Inputsummary.
Exercises.
Theprintstatement.
Simpleexamplesofprintstatements.
Theoutputseparatorsandhowtochangethem.
ControllingNumericOutputWithprint.
Theprintfstatement.
Syntaxoftheprintfstatement.
Formatcontrolletters.
Formatspecificationmodifiers.
Severalexamples.
Howtoredirectoutputtomultiplefilesandpipes.
SpecialfilesforI/O.
Filenameinterpretationingawk.gawkallowsaccesstoinheritedfiledescriptors.
Accessingotheropenfileswithgawk.
Specialfilesfornetworkcommunications.
Thingstowatchoutfor.
ClosingInputandOutputFilesandPipes.
Outputsummary.
3/479
29/7/2016
Constants:
ScalarConstants:
Nondecimalnumbers:
RegexpConstants:
UsingConstantRegexps:
Variables:
UsingVariables:
AssignmentOptions:
Conversion:
StringsAndNumbers:
Localeinfluences
conversions:
AllOperators:
ArithmeticOps:
Concatenation:
AssignmentOps:
IncrementOps:
TruthValuesand
Conditions:
TruthValues:
TypingandComparison:
TheGNUAwkUsersGuide
String,numericandregexpconstants.
Numericandstringconstants.
Whatareoctalandhexnumbers.
RegularExpressionconstants.
Whenandhowtousearegexpconstant.
Variablesgivenamestovaluesforlateruse.
Usingvariablesinyourprograms.
Settingvariablesonthecommandlineandasummaryofcommandlinesyntax.Thisis
anadvancedmethodofinput.
Theconversionofstringstonumbersandviceversa.
HowawkConvertsBetweenStringsAndNumbers.
Howthelocalemayaffectconversions.
gawksoperators.
Arithmeticoperations(+,,etc.)
Concatenatingstrings.
Changingthevalueofavariableorafield.
Incrementingthenumericvalueofavariable.
Testingfortrueandfalse.
Whatistrueandwhatisfalse.
Howvariablesacquiretypesandhowthisaffectscomparisonofnumbersandstrings
with<,etc.
VariableTyping:
Stringtypeversusnumerictype.
ComparisonOperators:
Thecomparisonoperators.
POSIXStringComparison: StringcomparisonwithPOSIXrules.
BooleanOps:
Combiningcomparisonexpressionsusingbooleanoperators||(or),&&(and)
and!(not).
ConditionalExp:
Conditionalexpressionsselectbetweentwosubexpressionsundercontrolofathird
subexpression.
FunctionCalls:
Afunctioncallisanexpression.
Precedence:
Howvariousoperatorsnest.
Locales:
Howthelocaleaffectsthings.
ExpressionsSummary:
Expressionssummary.
PatternOverview:
Whatgoesintoapattern.
RegexpPatterns:
Usingregexpsaspatterns.
ExpressionPatterns:
Anyexpressioncanbeusedasapattern.
Ranges:
Pairsofpatternsspecifyrecordranges.
BEGIN/END:
Specifyinginitializationandcleanuprules.
UsingBEGIN/END:
HowandwhytouseBEGIN/ENDrules.
I/OAndBEGIN/END:
I/OissuesinBEGIN/ENDrules.
BEGINFILE/ENDFILE:
Twospecialpatternsforadvancedcontrol.
Empty:
Theemptypattern,whichmatcheseveryrecord.
UsingShellVariables:
Howtouseshellvariableswithawk.
ActionOverview:
Whatgoesintoanaction.
Statements:
Describesthevariouscontrolstatementsindetail.
IfStatement:
Conditionallyexecutesomeawkstatements.
WhileStatement:
Loopuntilsomeconditionissatisfied.
DoStatement:
Dospecifiedactionwhileloopinguntilsomeconditionissatisfied.
ForStatement:
Anotherloopingstatement,thatprovidesinitializationandincrementclauses.
SwitchStatement:
Switch/caseevaluationforconditionalexecutionofstatementsbasedonavalue.
BreakStatement:
Immediatelyexittheinnermostenclosingloop.
4/479
29/7/2016
ContinueStatement:
NextStatement:
NextfileStatement:
ExitStatement:
BuiltinVariables:
Usermodified:
Autoset:
ARGCandARGV:
PatternActionSummary:
ArrayBasics:
ArrayIntro:
ReferencetoElements:
AssigningElements:
ArrayExample:
ScanninganArray:
ControllingScanning:
NumericArraySubscripts:
UninitializedSubscripts:
Delete:
Multidimensional:
Multiscanning:
ArraysofArrays:
ArraysSummary:
Builtin:
CallingBuiltin:
NumericFunctions:
StringFunctions:
GoryDetails:
I/OFunctions:
TimeFunctions:
BitwiseFunctions:
TypeFunctions:
I18NFunctions:
Userdefined:
DefinitionSyntax:
FunctionExample:
FunctionCaveats:
CallingAFunction:
VariableScope:
PassByValue/Reference:
ReturnStatement:
DynamicTyping:
IndirectCalls:
FunctionsSummary:
LibraryNames:
GeneralFunctions:
StrtonumFunction:
AssertFunction:
RoundFunction:
CliffRandomFunction:
OrdinalFunctions:
TheGNUAwkUsersGuide
Skiptotheendoftheinnermostenclosingloop.
Stopprocessingthecurrentinputrecord.
Stopprocessingthecurrentfile.
Stopexecutionofawk.
Summarizesthepredefinedvariables.
Builtinvariablesthatyouchangetocontrolawk.
Builtinvariableswhereawkgivesyouinformation.
WaystouseARGCandARGV.
PatternsandActionssummary.
Thebasicsofarrays.
IntroductiontoArrays
Howtoexamineoneelementofanarray.
Howtochangeanelementofanarray.
BasicExampleofanArray
Avariationoftheforstatement.Itloopsthroughtheindicesofanarraysexisting
elements.
Controllingtheorderinwhicharraysarescanned.
Howtousenumbersassubscriptsinawk.
UsingUninitializedvariablesassubscripts.
Thedeletestatementremovesanelementfromanarray.
Emulatingmultidimensionalarraysinawk.
Scanningmultidimensionalarrays.
Truemultidimensionalarrays.
Summaryofarrays.
Summarizesthebuiltinfunctions.
Howtocallbuiltinfunctions.
Functionsthatworkwithnumbers,includingint(),sin()andrand().
Functionsforstringmanipulation,suchassplit(),match()andsprintf().
Morethanyouwanttoknowabout\and&withsub(),gsub(),andgensub().
Functionsforfilesandshellcommands.
Functionsfordealingwithtimestamps.
Functionsforbitwiseoperations.
Functionsfortypeinformation.
Functionsforstringtranslation.
DescribesUserdefinedfunctionsindetail.
Howtowritedefinitionsandwhattheymean.
Anexamplefunctiondefinitionandwhatitdoes.
Dontusespaces.
Controllingvariablescope.
Passingparameters.
Specifyingthevalueafunctionreturns.
Howvariabletypescanchangeatruntime.
Choosingthefunctiontocallatruntime.
Summaryoffunctions.
Howtobestnameprivateglobalvariablesinlibraryfunctions.
Functionsthatareofgeneraluse.
Areplacementforthebuiltinstrtonum()function.
Afunctionforassertionsinawkprograms.
Afunctionforroundingifsprintf()doesnotdoitcorrectly.
TheCliffRandomNumberGenerator.
Functionsforusingcharactersasnumbersandviceversa.
5/479
29/7/2016
TheGNUAwkUsersGuide
JoinFunction:
Afunctiontojoinanarrayintoastring.
GetlocaltimeFunction:
Afunctiontogetformattedtimes.
ReadfileFunction:
Afunctiontoreadanentirefileatonce.
ShellQuoting:
Afunctiontoquotestringsfortheshell.
DataFileManagement:
Functionsformanagingcommandlinedatafiles.
FiletransFunction:
Afunctionforhandlingdatafiletransitions.
RewindFunction:
Afunctionforrereadingthecurrentfile.
FileChecking:
Checkingthatdatafilesarereadable.
EmptyFiles:
Checkingforzerolengthfiles.
IgnoringAssigns:
Treatingassignmentsasfilenames.
GetoptFunction:
Afunctionforprocessingcommandlinearguments.
PasswdFunctions:
Functionsforgettinguserinformation.
GroupFunctions:
Functionsforgettinggroupinformation.
WalkingArrays:
Afunctiontowalkarraysofarrays.
LibraryFunctionsSummary: Summaryoflibraryfunctions.
LibraryExercises:
Exercises.
RunningExamples:
Howtoruntheseexamples.
Clones:
Clonesofcommonutilities.
CutProgram:
Thecututility.
EgrepProgram:
Theegreputility.
IdProgram:
Theidutility.
SplitProgram:
Thesplitutility.
TeeProgram:
Theteeutility.
UniqProgram:
Theuniqutility.
WcProgram:
Thewcutility.
MiscellaneousPrograms:
Someinterestingawkprograms.
DupwordProgram:
Findingduplicatedwordsinadocument.
AlarmProgram:
Analarmclock.
TranslateProgram:
Aprogramsimilartothetrutility.
LabelsProgram:
Printingmailinglabels.
WordSorting:
Aprogramtoproduceawordusagecount.
HistorySorting:
Eliminatingduplicateentriesfromahistoryfile.
ExtractProgram:
PullingoutprogramsfromTexinfosourcefiles.
SimpleSed:
ASimpleStreamEditor.
IgawkProgram:
Awrapperforawkthatincludesfiles.
AnagramProgram:
Findinganagramsfromadictionary.
SignatureProgram:
Peopledoamazingthingswithtoomuchtimeontheirhands.
ProgramsSummary:
Summaryofprograms.
ProgramsExercises:
Exercises.
NondecimalData:
Allowingnondecimalinputdata.
ArraySorting:
Facilitiesforcontrollingarraytraversalandsortingarrays.
ControllingArrayTraversal: HowtousePROCINFO["sorted_in"].
ArraySortingFunctions:
Howtouseasort()andasorti().
TwowayI/O:
Twowaycommunicationswithanotherprocess.
TCP/IPNetworking:
Usinggawkfornetworkprogramming.
Profiling:
Profilingyourawkprograms.
AdvancedFeatures
Summaryofadvancedfeatures.
Summary:
I18NandL10N:
Explaininggettext:
Programmeri18n:
InternationalizationandLocalization.
HowGNUgettextworks.
Featuresfortheprogrammer.
6/479
29/7/2016
TheGNUAwkUsersGuide
Translatori18n:
Featuresforthetranslator.
StringExtraction:
Extractingmarkedstrings.
PrintfOrdering:
Rearrangingprintfarguments.
I18NPortability:
awklevelportabilityissues.
I18NExample:
Asimplei18nexample.
GawkI18N:
gawkisalsointernationalized.
I18NSummary:
SummaryofI18Nstuff.
Debugging:
Introductiontogawkdebugger.
DebuggingConcepts:
DebugginginGeneral.
DebuggingTerms:
AdditionalDebuggingConcepts.
AwkDebugging:
AwkDebugging.
SampleDebuggingSession: Sampledebuggingsession.
DebuggerInvocation:
HowtoStarttheDebugger.
FindingTheBug:
FindingtheBug.
ListofDebugger
Maindebuggercommands.
Commands:
BreakpointControl:
ControlofBreakpoints.
DebuggerExecution
ControlofExecution.
Control:
ViewingAndChanging
ViewingandChangingData.
Data:
ExecutionStack:
DealingwiththeStack.
DebuggerInfo:
ObtainingInformationabouttheProgramandtheDebuggerState.
MiscellaneousDebugger
MiscellaneousCommands.
Commands:
ReadlineSupport:
Readlinesupport.
Limitations:
Limitationsandfutureplans.
DebuggingSummary:
Debuggingsummary.
ComputerArithmetic:
Aquickintrotocomputermath.
MathDefinitions:
Definingtermsused.
MPFRfeatures:
TheMPFRfeaturesingawk.
FPMathCaution:
Thingstoknow.
Inexactnessofcomputations: Floatingpointmathisnotexact.
Inexactrepresentation:
Numbersarenotexactlyrepresented.
ComparingFPValues:
Howtocomparefloatingpointvalues.
Errorsaccumulate:
Errorsgetbiggerastheygo.
GettingAccuracy:
Gettingmoreaccuracytakessomework.
TryToRound:
Adddigitsandround.
Settingprecision:
Howtosettheprecision.
Settingtheroundingmode: Howtosettheroundingmode.
ArbitraryPrecisionIntegers: ArbitraryPrecisionIntegerArithmeticwithgawk.
POSIXFloatingPoint
StandardsVersusExistingPractice.
Problems:
Floatingpointsummary:
Summaryoffloatingpointdiscussion.
ExtensionIntro:
Whatisanextension.
PluginLicense:
Anoteaboutlicensing.
ExtensionMechanism
Anoutlineofhowitworks.
Outline:
ExtensionAPIDescription: AfulldescriptionoftheAPI.
ExtensionAPIFunctions
IntroductiontotheAPIfunctions.
Introduction:
GeneralDataTypes:
Thedatatypes.
7/479
29/7/2016
TheGNUAwkUsersGuide
MemoryAllocation
Functionsforallocatingmemory.
Functions:
ConstructorFunctions:
Functionsforcreatingvalues.
RegistrationFunctions:
Functionstoregisterthingswithgawk.
ExtensionFunctions:
Registeringextensionfunctions.
ExitCallbackFunctions:
Registeringanexitcallback.
ExtensionVersionString:
Registeringaversionstring.
InputParsers:
Registeringaninputparser.
OutputWrappers:
Registeringanoutputwrapper.
Twowayprocessors:
Registeringatwowayprocessor.
PrintingMessages:
Functionsforprintingmessages.
UpdatingERRNO:
FunctionsforupdatingERRNO.
RequestingValues:
Howtogetavalue.
AccessingParameters:
Functionsforaccessingparameters.
SymbolTableAccess:
Functionsforaccessingglobalvariables.
Symboltablebyname:
Accessingvariablesbyname.
Symboltablebycookie:
Accessingvariablesbycookie.
Cachedvalues:
Creatingandusingcachedvalues.
ArrayManipulation:
Functionsforworkingwitharrays.
ArrayDataTypes:
Datatypesforworkingwitharrays.
ArrayFunctions:
FlatteningArrays:
Howtoflattenarrays.
CreatingArrays:
Howtocreateandpopulatearrays.
ExtensionAPIVariables:
VariablesprovidedbytheAPI.
ExtensionVersioning:
APIVersioninformation.
ExtensionAPIInformational Variablesprovidinginformationaboutgawksinvocation.
Variables:
ExtensionAPIBoilerplate: BoilerplatecodeforusingtheAPI.
FindingExtensions:
Howgawkfindscompiledextensions.
ExtensionExample:
ExampleCcodeforanextension.
InternalFileDescription:
Whatthenewfunctionswilldo.
InternalFileOps:
Thecodeforinternalfileoperations.
UsingInternalFileOps:
Howtouseanexternalextension.
ExtensionSamples:
Thesampleextensionsthatshipwithgawk.
ExtensionSampleFile
Thefilefunctionssample.
Functions:
ExtensionSampleFnmatch: Aninterfacetofnmatch().
ExtensionSampleFork:
Aninterfacetofork()andotherprocessfunctions.
ExtensionSampleInplace: Enablinginplacefileediting.
ExtensionSampleOrd:
Charactertovaluetocharacterconversions.
ExtensionSampleReaddir: Aninterfacetoreaddir().
ExtensionSampleRevout: Reversingoutputsampleoutputwrapper.
ExtensionSampleRev2way: Reversingdatasampletwowayprocessor.
ExtensionSampleRead
Serializinganarraytoafile.
writearray:
ExtensionSampleReadfile: Readinganentirefileintoastring.
ExtensionSampleTime:
Aninterfacetogettimeofday()andsleep().
ExtensionSampleAPITests: TestsfortheAPI.
gawkextlib:
Thegawkextlibproject.
Extensionsummary:
Extensionsummary.
ExtensionExercises:
Exercises.
V7/SVR3.1:
ThemajorchangesbetweenV7andSystemVRelease3.1.
8/479
29/7/2016
TheGNUAwkUsersGuide
SVR4:
POSIX:
BTL:
POSIX/GNU:
FeatureHistory:
CommonExtensions:
RangesandLocales:
Contributors:
Historysummary:
GawkDistribution:
Getting:
Extracting:
Distributioncontents:
UnixInstallation:
QuickInstallation:
AdditionalConfiguration
Options:
ConfigurationPhilosophy:
NonUnixInstallation:
PCInstallation:
PCBinaryInstallation:
PCCompiling:
PCTesting:
PCUsing:
Cygwin:
MSYS:
VMSInstallation:
VMSCompilation:
VMSDynamicExtensions:
VMSInstallationDetails:
VMSRunning:
VMSGNV:
VMSOldGawk:
Bugs:
OtherVersions:
Installationsummary:
CompatibilityMode:
MinorchangesbetweenSystemVReleases3.1and4.
NewfeaturesfromthePOSIXstandard.
NewfeaturesfromBrianKernighansversionofawk.
TheextensionsingawknotinPOSIXawk.
Thehistoryofthefeaturesingawk.
CommonExtensionsSummary.
Howlocalesusedtoaffectregexpranges.
Themajorcontributorstogawk.
Historysummary.
Whatisinthegawkdistribution.
Howtogetthedistribution.
Howtoextractthedistribution.
Whatisinthedistribution.
InstallinggawkundervariousversionsofUnix.
CompilinggawkunderUnix.
Othercompiletimeoptions.
Additions:
AccessingTheSource:
AddingCode:
NewPorts:
DerivedFiles:
FutureExtensions:
ImplementationLimitations:
ExtensionDesign:
OldExtensionProblems:
ExtensionNewMechanism
Goals:
ExtensionOtherDesign
Decisions:
ExtensionFutureGrowth:
OldExtensionMechanism:
MakingAdditionsTogawk.
AccessingtheGitrepository.
Addingcodetothemainbodyofgawk.
Portinggawktoanewoperatingsystem.
WhyderivedfilesarekeptintheGitrepository.
Newfeaturesthatmaybeimplementedoneday.
Somelimitationsoftheimplementation.
DesignnotesabouttheextensionAPI.
Problemswiththeoldmechanism.
Goalsforthenewmechanism.
Howitsallsupposedtowork.
InstallationonOtherOperatingSystems.
InstallingandCompilinggawkonMSDOSandOS/2.
Installingaprepareddistribution.
CompilinggawkforMSDOS,Windows32,andOS/2.
TestinggawkonPCsystems.
RunninggawkonMSDOS,Windows32andOS/2.
BuildingandrunninggawkforCygwin.
UsinggawkInTheMSYSEnvironment.
InstallinggawkonVMS.
HowtocompilegawkunderVMS.
CompilinggawkdynamicextensionsonVMS.
HowtoinstallgawkunderVMS.
HowtorungawkunderVMS.
TheVMSGNVProject.
AnoldversioncomeswithsomeVMSsystems.
ReportingProblemsandBugs.
Otherfreelyavailableawkimplementations.
Summaryofinstallation.
Howtodisablecertaingawkextensions.
Someotherdesigndecisions.
Someroomforfuturegrowth.
Somecompatibilityforoldextensions.
9/479
29/7/2016
TheGNUAwkUsersGuide
Notessummary:
BasicHighLevel:
BasicDataTyping:
Summaryofimplementationnotes.
Thehighlevelview.
Averyquickintrotodatatypes.
ShortTableofContents
ForewordtotheThirdEdition
ForewordtotheFourthEdition
Preface
PartI:
TheawkLanguage
1GettingStartedwithawk
2Runningawkandgawk
3RegularExpressions
4ReadingInputFiles
5PrintingOutput
6Expressions
7Patterns,Actions,andVariables
8Arraysinawk
9Functions
PartII:
ProblemSolvingwithawk
10ALibraryofawkFunctions
11PracticalawkPrograms
PartIII:
MovingBeyondStandardawkwithgawk
12AdvancedFeaturesofgawk
13Internationalizationwithgawk
14DebuggingawkPrograms
15ArithmeticandArbitraryPrecisionArithmeticwithgawk
16WritingExtensionsforgawk
PartIV:
Appendices
AppendixATheEvolutionoftheawkLanguage
AppendixBInstallinggawk
AppendixCImplementationNotes
AppendixDBasicProgrammingConcepts
Glossary
GNUGeneralPublicLicense
GNUFreeDocumentationLicense
Index
TableofContents
Preface
Historyofawkandgawk
ARosebyAnyOtherName
UsingThisBook
TypographicalConventions
DarkCorners
TheGNUProjectandThisBook
HowtoContribute
Acknowledgments
PartI:
10/479
29/7/2016
TheGNUAwkUsersGuide
TheawkLanguage
1.1HowtoRunawkPrograms
1.1.1OneShotThrowawayawkPrograms
1.1.2RunningawkWithoutInputFiles
1.1.3RunningLongPrograms
1.1.4ExecutableawkPrograms
1.1.5CommentsinawkPrograms
1.1.6ShellQuotingIssues
1.1.6.1QuotinginMSWindowsBatchFiles
1.2DatafilesfortheExamples
1.3SomeSimpleExamples
1.4AnExamplewithTwoRules
1.5AMoreComplexExample
1.6awkStatementsVersusLines
1.7OtherFeaturesofawk
1.8WhentoUseawk
1.9Summary
2Runningawkandgawk
2.1Invokingawk
2.2CommandLineOptions
2.3OtherCommandLineArguments
2.4NamingStandardInput
2.5TheEnvironmentVariablesgawkUses
2.5.1TheAWKPATHEnvironmentVariable
2.5.2TheAWKLIBPATHEnvironmentVariable
2.5.3OtherEnvironmentVariables
2.6gawksExitStatus
2.7IncludingOtherFilesintoYourProgram
2.8LoadingDynamicExtensionsintoYourProgram
2.9ObsoleteOptionsand/orFeatures
2.10UndocumentedOptionsandFeatures
2.11Summary
3RegularExpressions
3.1HowtoUseRegularExpressions
3.2EscapeSequences
3.3RegularExpressionOperators
3.4UsingBracketExpressions
3.5HowMuchTextMatches?
3.6UsingDynamicRegexps
3.7gawkSpecificRegexpOperators
3.8CaseSensitivityinMatching
3.9Summary
4ReadingInputFiles
4.1HowInputIsSplitintoRecords
4.1.1RecordSplittingwithStandardawk
4.1.2RecordSplittingwithgawk
4.2ExaminingFields
4.3NonconstantFieldNumbers
4.4ChangingtheContentsofaField
4.5SpecifyingHowFieldsAreSeparated
4.5.1WhitespaceNormallySeparatesFields
4.5.2UsingRegularExpressionstoSeparateFields
4.5.3MakingEachCharacteraSeparateField
4.5.4SettingFSfromtheCommandLine
4.5.5MakingtheFullLineBeaSingleField
4.5.6FieldSplittingSummary
11/479
29/7/2016
TheGNUAwkUsersGuide
4.6ReadingFixedWidthData
4.7DefiningFieldsbyContent
4.8MultipleLineRecords
4.9ExplicitInputwithgetline
4.9.1UsinggetlinewithNoArguments
4.9.2UsinggetlineintoaVariable
4.9.3UsinggetlinefromaFile
4.9.4UsinggetlineintoaVariablefromaFile
4.9.5UsinggetlinefromaPipe
4.9.6UsinggetlineintoaVariablefromaPipe
4.9.7UsinggetlinefromaCoprocess
4.9.8UsinggetlineintoaVariablefromaCoprocess
4.9.9PointstoRememberAboutgetline
4.9.10SummaryofgetlineVariants
4.10ReadingInputwithaTimeout
4.11DirectoriesontheCommandLine
4.12Summary
4.13Exercises
5PrintingOutput
5.1TheprintStatement
5.2printStatementExamples
5.3OutputSeparators
5.4ControllingNumericOutputwithprint
5.5UsingprintfStatementsforFancierPrinting
5.5.1IntroductiontotheprintfStatement
5.5.2FormatControlLetters
5.5.3ModifiersforprintfFormats
5.5.4ExamplesUsingprintf
5.6RedirectingOutputofprintandprintf
5.7SpecialFilesforStandardPreopenedDataStreams
5.8SpecialFilenamesingawk
5.8.1AccessingOtherOpenFileswithgawk
5.8.2SpecialFilesforNetworkCommunications
5.8.3SpecialFilenameCaveats
5.9ClosingInputandOutputRedirections
5.10Summary
5.11Exercises
6Expressions
6.1Constants,Variables,andConversions
6.1.1ConstantExpressions
6.1.1.1NumericandStringConstants
6.1.1.2OctalandHexadecimalNumbers
6.1.1.3RegularExpressionConstants
6.1.2UsingRegularExpressionConstants
6.1.3Variables
6.1.3.1UsingVariablesinaProgram
6.1.3.2AssigningVariablesontheCommandLine
6.1.4ConversionofStringsandNumbers
6.1.4.1HowawkConvertsBetweenStringsandNumbers
6.1.4.2LocalesCanInfluenceConversion
6.2Operators:DoingSomethingwithValues
6.2.1ArithmeticOperators
6.2.2StringConcatenation
6.2.3AssignmentExpressions
6.2.4IncrementandDecrementOperators
6.3TruthValuesandConditions
6.3.1TrueandFalseinawk
12/479
29/7/2016
TheGNUAwkUsersGuide
6.3.2VariableTypingandComparisonExpressions
6.3.2.1StringTypeversusNumericType
6.3.2.2ComparisonOperators
6.3.2.3StringComparisonwithPOSIXRules
6.3.3BooleanExpressions
6.3.4ConditionalExpressions
6.4FunctionCalls
6.5OperatorPrecedence(HowOperatorsNest)
6.6WhereYouAreMakesaDifference
6.7Summary
7.1PatternElements
7.1.1RegularExpressionsasPatterns
7.1.2ExpressionsasPatterns
7.1.3SpecifyingRecordRangeswithPatterns
7.1.4TheBEGINandENDSpecialPatterns
7.1.4.1StartupandCleanupActions
7.1.4.2Input/OutputfromBEGINandENDRules
7.1.5TheBEGINFILEandENDFILESpecialPatterns
7.1.6TheEmptyPattern
7.2UsingShellVariablesinPrograms
7.3Actions
7.4ControlStatementsinActions
7.4.1TheifelseStatement
7.4.2ThewhileStatement
7.4.3ThedowhileStatement
7.4.4TheforStatement
7.4.5TheswitchStatement
7.4.6ThebreakStatement
7.4.7ThecontinueStatement
7.4.8ThenextStatement
7.4.9ThenextfileStatement
7.4.10TheexitStatement
7.5PredefinedVariables
7.5.1BuiltinVariablesThatControlawk
7.5.2BuiltinVariablesThatConveyInformation
7.5.3UsingARGCandARGV
7.6Summary
8Arraysinawk
8.1TheBasicsofArrays
8.1.1IntroductiontoArrays
8.1.2ReferringtoanArrayElement
8.1.3AssigningArrayElements
8.1.4BasicArrayExample
8.1.5ScanningAllElementsofanArray
8.1.6UsingPredefinedArrayScanningOrderswithgawk
8.2UsingNumberstoSubscriptArrays
8.3UsingUninitializedVariablesasSubscripts
8.4ThedeleteStatement
8.5MultidimensionalArrays
8.5.1ScanningMultidimensionalArrays
8.6ArraysofArrays
8.7Summary
9Functions
9.1BuiltinFunctions
9.1.1CallingBuiltinFunctions
9.1.2NumericFunctions
13/479
29/7/2016
TheGNUAwkUsersGuide
9.1.3StringManipulationFunctions
9.1.3.1Moreabout\and&withsub(),gsub(),andgensub()
9.1.4Input/OutputFunctions
9.1.5TimeFunctions
9.1.6BitManipulationFunctions
9.1.7GettingTypeInformation
9.1.8StringTranslationFunctions
9.2UserDefinedFunctions
9.2.1FunctionDefinitionSyntax
9.2.2FunctionDefinitionExamples
9.2.3CallingUserDefinedFunctions
9.2.3.1WritingaFunctionCall
9.2.3.2ControllingVariableScope
9.2.3.3PassingFunctionArgumentsbyValueOrbyReference
9.2.4ThereturnStatement
9.2.5FunctionsandTheirEffectsonVariableTyping
9.3IndirectFunctionCalls
9.4Summary
PartII:
10.1NamingLibraryFunctionGlobalVariables
10.2GeneralProgramming
10.2.1ConvertingStringstoNumbers
10.2.2Assertions
10.2.3RoundingNumbers
10.2.4TheCliffRandomNumberGenerator
10.2.5TranslatingBetweenCharactersandNumbers
10.2.6MerginganArrayintoaString
10.2.7ManagingtheTimeofDay
10.2.8ReadingaWholeFileatOnce
10.2.9QuotingStringstoPasstotheShell
10.3DatafileManagement
10.3.1NotingDatafileBoundaries
10.3.2RereadingtheCurrentFile
10.3.3CheckingforReadableDatafiles
10.3.4CheckingforZeroLengthFiles
10.3.5TreatingAssignmentsasFilenames
10.4ProcessingCommandLineOptions
10.5ReadingtheUserDatabase
10.6ReadingtheGroupDatabase
10.7TraversingArraysofArrays
10.8Summary
10.9Exercises
11.1RunningtheExamplePrograms
11.2ReinventingWheelsforFunandProfit
11.2.1CuttingOutFieldsandColumns
11.2.2SearchingforRegularExpressionsinFiles
11.2.3PrintingOutUserInformation
11.2.4SplittingaLargeFileintoPieces
11.2.5DuplicatingOutputintoMultipleFiles
11.2.6PrintingNonduplicatedLinesofText
11.2.7CountingThings
11.3AGrabBagofawkPrograms
11.3.1FindingDuplicatedWordsinaDocument
11.3.2AnAlarmClockProgram
14/479
29/7/2016
TheGNUAwkUsersGuide
11.3.3TransliteratingCharacters
11.3.4PrintingMailingLabels
11.3.5GeneratingWordUsageCounts
11.3.6RemovingDuplicatesfromUnsortedText
11.3.7ExtractingProgramsfromTexinfoSourceFiles
11.3.8ASimpleStreamEditor
11.3.9AnEasyWaytoUseLibraryFunctions
11.3.10FindingAnagramsfromaDictionary
11.3.11AndNowforSomethingCompletelyDifferent
11.4Summary
11.5Exercises
PartIII:
12.1AllowingNondecimalInputData
12.2ControllingArrayTraversalandArraySorting
12.2.1ControllingArrayTraversal
12.2.2SortingArrayValuesandIndiceswithgawk
12.3TwoWayCommunicationswithAnotherProcess
12.4UsinggawkforNetworkProgramming
12.5ProfilingYourawkPrograms
12.6Summary
13.1InternationalizationandLocalization
13.2GNUgettext
13.3InternationalizingawkPrograms
13.4TranslatingawkPrograms
13.4.1ExtractingMarkedStrings
13.4.2RearrangingprintfArguments
13.4.3awkPortabilityIssues
13.5ASimpleInternationalizationExample
13.6gawkCanSpeakYourLanguage
13.7Summary
14.1IntroductiontothegawkDebugger
14.1.1DebugginginGeneral
14.1.2DebuggingConcepts
14.1.3awkDebugging
14.2SamplegawkDebuggingSession
14.2.1HowtoStarttheDebugger
14.2.2FindingtheBug
14.3MainDebuggerCommands
14.3.1ControlofBreakpoints
14.3.2ControlofExecution
14.3.3ViewingandChangingData
14.3.4WorkingwiththeStack
14.3.5ObtainingInformationAbouttheProgramandtheDebuggerState
14.3.6MiscellaneousCommands
14.4ReadlineSupport
14.5Limitations
14.6Summary
15.1AGeneralDescriptionofComputerArithmetic
15.2OtherStufftoKnow
15.3ArbitraryPrecisionArithmeticFeaturesingawk
15.4FloatingPointArithmetic:CaveatEmptor!
15.4.1FloatingPointArithmeticIsNotExact
15/479
29/7/2016
TheGNUAwkUsersGuide
15.4.1.1ManyNumbersCannotBeRepresentedExactly
15.4.1.2BeCarefulComparingValues
15.4.1.3ErrorsAccumulate
15.4.2GettingtheAccuracyYouNeed
15.4.3TryaFewExtraBitsofPrecisionandRounding
15.4.4SettingthePrecision
15.4.5SettingtheRoundingMode
15.5ArbitraryPrecisionIntegerArithmeticwithgawk
15.6StandardsVersusExistingPractice
15.7Summary
16.1Introduction
16.2ExtensionLicensing
16.3HowItWorksataHighLevel
16.4APIDescription
16.4.1Introduction
16.4.2GeneralPurposeDataTypes
16.4.3MemoryAllocationFunctionsandConvenienceMacros
16.4.4ConstructorFunctions
16.4.5RegistrationFunctions
16.4.5.1RegisteringAnExtensionFunction
16.4.5.2RegisteringAnExitCallbackFunction
16.4.5.3RegisteringAnExtensionVersionString
16.4.5.4CustomizedInputParsers
16.4.5.5CustomizedOutputWrappers
16.4.5.6CustomizedTwowayProcessors
16.4.6PrintingMessages
16.4.7UpdatingERRNO
16.4.8RequestingValues
16.4.9AccessingandUpdatingParameters
16.4.10SymbolTableAccess
16.4.10.1VariableAccessandUpdatebyName
16.4.10.2VariableAccessandUpdatebyCookie
16.4.10.3CreatingandUsingCachedValues
16.4.11ArrayManipulation
16.4.11.1ArrayDataTypes
16.4.11.2ArrayFunctions
16.4.11.3WorkingWithAllTheElementsofanArray
16.4.11.4HowToCreateandPopulateArrays
16.4.12APIVariables
16.4.12.1APIVersionConstantsandVariables
16.4.12.2InformationalVariables
16.4.13BoilerplateCode
16.5HowgawkFindsExtensions
16.6Example:SomeFileFunctions
16.6.1Usingchdir()andstat()
16.6.2CCodeforchdir()andstat()
16.6.3IntegratingtheExtensions
16.7TheSampleExtensionsinthegawkDistribution
16.7.1FileRelatedFunctions
16.7.2Interfacetofnmatch()
16.7.3Interfacetofork(),wait(),andwaitpid()
16.7.4EnablingInPlaceFileEditing
16.7.5CharacterandNumericvalues:ord()andchr()
16.7.6ReadingDirectories
16.7.7ReversingOutput
16.7.8TwoWayI/OExample
16/479
29/7/2016
TheGNUAwkUsersGuide
16.7.9DumpingandRestoringanArray
16.7.10ReadinganEntireFile
16.7.11ExtensionTimeFunctions
16.7.12APITests
16.8ThegawkextlibProject
16.9Summary
16.10Exercises
PartIV:
Appendices
A.1MajorChangesBetweenV7andSVR3.1
A.2ChangesBetweenSVR3.1andSVR4
A.3ChangesBetweenSVR4andPOSIXawk
A.4ExtensionsinBrianKernighansawk
A.5ExtensionsingawkNotinPOSIXawk
A.6HistoryofgawkFeatures
A.7CommonExtensionsSummary
A.8RegexpRangesandLocales:ALongSadStory
A.9MajorContributorstogawk
A.10Summary
B.1ThegawkDistribution
B.1.1GettingthegawkDistribution
B.1.2ExtractingtheDistribution
B.1.3ContentsofthegawkDistribution
B.2CompilingandInstallinggawkonUnixLikeSystems
B.2.1CompilinggawkforUnixLikeSystems
B.2.2AdditionalConfigurationOptions
B.2.3TheConfigurationProcess
B.3InstallationonOtherOperatingSystems
B.3.1InstallationonPCOperatingSystems
B.3.1.1InstallingaPreparedDistributionforPCSystems
B.3.1.2CompilinggawkforPCOperatingSystems
B.3.1.3TestinggawkonPCOperatingSystems
B.3.1.4UsinggawkonPCOperatingSystems
B.3.1.5UsinggawkInTheCygwinEnvironment
B.3.1.6UsinggawkInTheMSYSEnvironment
B.3.2CompilingandInstallinggawkonVax/VMSandOpenVMS
B.3.2.1CompilinggawkonVMS
B.3.2.2CompilinggawkDynamicExtensionsonVMS
B.3.2.3InstallinggawkonVMS
B.3.2.4RunninggawkonVMS
B.3.2.5TheVMSGNVProject
B.3.2.6SomeVMSSystemsHaveAnOldVersionofgawk
B.4ReportingProblemsandBugs
B.5OtherFreelyAvailableawkImplementations
B.6Summary
C.1DownwardCompatibilityandDebugging
C.2MakingAdditionstogawk
C.2.1AccessingThegawkGitRepository
C.2.2AddingNewFeatures
C.2.3PortinggawktoaNewOperatingSystem
C.2.4WhyGeneratedFilesAreKeptInGit
C.3ProbableFutureExtensions
C.4SomeLimitationsoftheImplementation
C.5ExtensionAPIDesign
17/479
29/7/2016
TheGNUAwkUsersGuide
C.5.1ProblemsWithTheOldMechanism
C.5.2GoalsForANewMechanism
C.5.3OtherDesignDecisions
C.5.4RoomForFutureGrowth
C.6CompatibilityForOldExtensions
C.7Summary
D.1WhataProgramDoes
D.2DataValuesinaComputer
Glossary
ADDENDUM:HowtousethisLicenseforyourdocuments
Index
Next:Foreword4,Previous:Top,Up:Top[Contents][Index]
ArnoldRobbinsandIaregoodfriends.Wewereintroducedin1990bycircumstancesandourfavoriteprogramming
language,AWK.Thecircumstancesstartedacoupleofyearsearlier.Iwasworkingatanewjobandnoticedan
unpluggedUnixcomputersittinginthecorner.Nooneknewhowtouseit,andneitherdidI.However,acoupleofdays
later,itwasrunning,andIwasrootandtheoneandonlyuser.Thatday,IbeganthetransitionfromstatisticiantoUnix
programmer.
OnoneofmanytripstothelibraryorbookstoreinsearchofbooksonUnix,IfoundthegrayAWKbook,a.k.a.Alfred
V.Aho,BrianW.Kernighan,andPeterJ.WeinbergersTheAWKProgrammingLanguage(AddisonWesley,1988).
awkssimpleprogrammingparadigmfindapatternintheinputandthenperformanactionoftenreducedcomplexor
tediousdatamanipulationstoafewlinesofcode.IwasexcitedtotrymyhandatprogramminginAWK.
Alas,theawkonmycomputerwasalimitedversionofthelanguagedescribedinthegraybook.Idiscoveredthatmy
computerhadoldawkandthebookdescribednewawk.Ilearnedthatthiswastypicaltheoldversionrefusedtostep
asideorrelinquishitsname.Ifasystemhadanewawk,itwasinvariablycallednawk,andfewsystemshadit.Thebest
waytogetanewawkwastoftpthesourcecodeforgawkfromprep.ai.mit.edu.gawkwasaversionofnewawkwritten
byDavidTruemanandArnold,andavailableundertheGNUGeneralPublicLicense.
(Incidentally,itsnolongerdifficulttofindanewawk.gawkshipswithGNU/Linux,andyoucandownloadbinariesor
sourcecodeforalmostanysystemmywifeusesgawkonherVMSbox.)
MyUnixsystemstartedoutunpluggedfromthewallitcertainlywasnotpluggedintoanetwork.So,oblivioustothe
existenceofgawkandtheUnixcommunityingeneral,anddesiringanewawk,Iwrotemyown,calledmawk.BeforeI
wasfinished,Iknewaboutgawk,butitwastoolatetostop,soIeventuallypostedtoacomp.sourcesnewsgroup.
Afewdaysaftermyposting,IgotafriendlyemailfromArnoldintroducinghimself.Hesuggestedwesharedesignand
algorithmsandattachedadraftofthePOSIXstandardsothatIcouldupdatemawktosupportlanguageextensionsadded
afterpublicationofTheAWKProgrammingLanguage.
Frankly,ifourroleshadbeenreversed,Iwouldnothavebeensoopenandweprobablywouldhavenevermet.Imglad
wedidmeet.HeisanAWKexpertsAWKexpertandagenuinelyniceperson.Arnoldcontributessignificantamounts
ofhisexpertiseandtimetotheFreeSoftwareFoundation.
Thisbookisthegawkreferencemanual,butatitscoreitisabookaboutAWKprogrammingthatwillappealtoawide
audience.ItisadefinitivereferencetotheAWKlanguageasdefinedbythe1987BellLaboratoriesreleaseandcodified
inthe1992POSIXUtilitiesstandard.
Ontheotherhand,thenoviceAWKprogrammercanstudyawealthofpracticalprogramsthatemphasizethepowerof
AWKsbasicidioms:datadrivencontrolflow,patternmatchingwithregularexpressions,andassociativearrays.Those
18/479
29/7/2016
TheGNUAwkUsersGuide
lookingforsomethingnewcantryoutgawksinterfacetonetworkprotocolsviaspecial/inetfiles.
TheprogramsinthisbookmakeclearthatanAWKprogramistypicallymuchsmallerandfastertodevelopthana
counterpartwritteninC.Consequently,thereisoftenapayofftoprototypinganalgorithmordesigninAWKtogetit
runningquicklyandexposeproblemsearly.Often,theinterpretedperformanceisadequateandtheAWKprototype
becomestheproduct.
Thenewpgawk(profilinggawk),producesprogramexecutioncounts.Irecentlyexperimentedwithanalgorithmthatfor
nlinesofinput,exhibited~Cn^2performance,whiletheorypredicted~Cnlognbehavior.Afewminutesporingover
theawkprof.outprofilepinpointedtheproblemtoasinglelineofcode.pgawkisawelcomeadditiontomy
programmerstoolbox.
ArnoldhasdistilledoveradecadeofexperiencewritingandusingAWKprograms,anddevelopinggawk,intothisbook.
IfyouuseAWKorwanttolearnhow,thenreadthisbook.
MichaelBrennan
Authorofmawk
March2001
Next:Preface,Previous:Foreword3,Up:Top[Contents][Index]
Somethingsdontchange.ThirteenyearsagoIwrote:IfyouuseAWKorwanttolearnhow,thenreadthisbook.
Truethen,andstilltruetoday.
Learningtouseaprogramminglanguageisaboutmorethanmasteringthesyntax.Oneneedstoacquirean
understandingofhowtousethefeaturesofthelanguagetosolvepracticalprogrammingproblems.Afocusofthisbook
ismanyexamplesthatshowhowtouseAWK.
Somethingsdochange.Ourcomputersaremuchfasterandhavemorememory.Consequently,speedandstorage
inefficienciesofahighlevellanguagematterless.PrototypinginAWKandthenrewritinginCforperformancereasons
happensless,becausemoreoftentheprototypeisfastenough.
Ofcourse,therearecomputingoperationsthatarebestdoneinCorC++.Withgawk4.1andlater,youdonothaveto
choosebetweenwritingyourprograminAWKorinC/C++.YoucanwritemostofyourprograminAWKandthe
aspectsthatrequireC/C++capabilitiescanbewritteninC/C++,andthenthepiecesgluedtogetherwhenthegawk
moduleloadstheC/C++moduleasadynamicplugin.DynamicExtensions,hasallthedetails,and,asexpected,many
examplestohelpyoulearntheinsandouts.
IenjoyprogramminginAWKandhadfun(re)readingthisbook.Ithinkyouwilltoo.
MichaelBrennan
Authorofmawk
October2014
Next:GettingStarted,Previous:Foreword4,Up:Top[Contents][Index]
Preface
Severalkindsoftasksoccurrepeatedlywhenworkingwithtextfiles.Youmightwanttoextractcertainlinesanddiscard
therest.Oryoumayneedtomakechangeswherevercertainpatternsappear,butleavetherestofthefilealone.Such
jobsareofteneasywithawk.Theawkutilityinterpretsaspecialpurposeprogramminglanguagethatmakesiteasyto
handlesimpledatareformattingjobs.
19/479
29/7/2016
TheGNUAwkUsersGuide
TheGNUimplementationofawkiscalledgawkifyouinvokeitwiththeproperoptionsorenvironmentvariables,itis
fullycompatiblewiththePOSIX1specificationoftheawklanguageandwiththeUnixversionofawkmaintainedby
BrianKernighan.Thismeansthatallproperlywrittenawkprogramsshouldworkwithgawk.Somostofthetime,we
dontdistinguishbetweengawkandotherawkimplementations.
Usingawkyoucan:
Managesmall,personaldatabases
Generatereports
Validatedata
Produceindexesandperformotherdocumentpreparationtasks
Experimentwithalgorithmsthatyoucanadaptlatertoothercomputerlanguages
Inaddition,gawkprovidesfacilitiesthatmakeiteasyto:
Extractbitsandpiecesofdataforprocessing
Sortdata
Performsimplenetworkcommunications
Profileanddebugawkprograms
ExtendthelanguagewithfunctionswritteninCorC++
ThisWebpageteachesyouabouttheawklanguageandhowyoucanuseiteffectively.Youshouldalreadybefamiliar
withbasicsystemcommands,suchascatandls,2aswellasbasicshellfacilities,suchasinput/output(I/O)redirection
andpipes.
Implementationsoftheawklanguageareavailableformanydifferentcomputingenvironments.ThisWebpage,while
describingtheawklanguageingeneral,alsodescribestheparticularimplementationofawkcalledgawk(whichstandsfor
GNUawk).gawkrunsonabroadrangeofUnixsystems,rangingfromIntelarchitecturePCbasedcomputersup
throughlargescalesystems.gawkhasalsobeenportedtoMacOSX,MicrosoftWindows(allversions)andOS/2PCs,
andOpenVMS.3
History:
Thehistoryofgawkandawk.
Names:
Whatnametousetofindawk.
ThisManual:
UsingthisWebpage.Includessampleinputfilesthatyoucanuse.
Conventions:
TypographicalConventions.
ManualHistory:
BriefhistoryoftheGNUprojectandthisWebpage.
HowToContribute: Helpingtosavetheworld.
Acknowledgments: Acknowledgments.
Next:Names,Up:Preface[Contents][Index]
Historyofawkandgawk
RecipeforaProgrammingLanguage
1partegrep 1partsnobol
2partsed 3partsC
Blendallpartswellusinglexandyacc.Documentminimallyandrelease.
Aftereightyears,addanotherpartegrepandtwomorepartsC.Documentverywellandrelease.
Thenameawkcomesfromtheinitialsofitsdesigners:AlfredV.Aho,PeterJ.Weinberger,andBrianW.Kernighan.The
originalversionofawkwaswrittenin1977atAT&TBellLaboratories.In1985,anewversionmadetheprogramming
languagemorepowerful,introducinguserdefinedfunctions,multipleinputstreams,andcomputedregularexpressions.
20/479
29/7/2016
TheGNUAwkUsersGuide
ThisnewversionbecamewidelyavailablewithUnixSystemVRelease3.1(1987).TheversioninSystemVRelease4
(1989)addedsomenewfeaturesandcleanedupthebehaviorinsomeofthedarkcornersofthelanguage.The
specificationforawkinthePOSIXCommandLanguageandUtilitiesstandardfurtherclarifiedthelanguage.Boththe
gawkdesignersandtheoriginalawkdesignersatBellLaboratoriesprovidedfeedbackforthePOSIXspecification.
PaulRubinwrotegawkin1986.JayFenlasoncompletedit,withadvicefromRichardStallman.JohnWoodscontributed
partsofthecodeaswell.In1988and1989,DavidTrueman,withhelpfromme,thoroughlyreworkedgawkfor
compatibilitywiththenewerawk.Circa1994,Ibecametheprimarymaintainer.Currentdevelopmentfocusesonbug
fixes,performanceimprovements,standardscompliance,and,occasionally,newfeatures.
InMay1997,JrgenKahrsfelttheneedfornetworkaccessfromawk,andwithalittlehelpfromme,setaboutadding
featurestodothisforgawk.Atthattime,healsowrotethebulkofTCP/IPInternetworkingwithgawk(aseparate
document,availableaspartofthegawkdistribution).Hiscodefinallybecamepartofthemaingawkdistributionwith
gawkversion3.1.
JohnHaquerewrotethegawkinternals,intheprocessprovidinganawkleveldebugger.Thisversionbecameavailableas
gawkversion4.0in2011.
SeeContributors,forafulllistofthosewhohavemadeimportantcontributionstogawk.
Next:ThisManual,Previous:History,Up:Preface[Contents][Index]
ARosebyAnyOtherName
Theawklanguagehasevolvedovertheyears.FulldetailsareprovidedinLanguageHistory.Thelanguagedescribedin
thisWebpageisoftenreferredtoasnewawk.Byanalogy,theoriginalversionofawkisreferredtoasoldawk.
Onmostcurrentsystems,whenyouruntheawkutilityyougetsomeversionofnewawk.4Ifyoursystemsstandardawk
istheoldone,youwillseesomethinglikethisifyoutrythetestprogram:
$awk1/dev/null
errorawk:syntaxerrornearline1
errorawk:bailingoutnearline1
Inthiscase,youshouldfindaversionofnewawk,orjustinstallgawk!
ThroughoutthisWebpage,wheneverwerefertoalanguagefeaturethatshouldbeavailableinanycomplete
implementationofPOSIXawk,wesimplyusethetermawk.WhenreferringtoafeaturethatisspecifictotheGNU
implementation,weusethetermgawk.
Next:Conventions,Previous:Names,Up:Preface[Contents][Index]
UsingThisBook
Thetermawkreferstoaparticularprogramaswellastothelanguageyouusetotellthisprogramwhattodo.Whenwe
needtobecareful,wecallthelanguagetheawklanguage,andtheprogramtheawkutility.ThisWebpageexplains
bothhowtowriteprogramsintheawklanguageandhowtoruntheawkutility.Thetermawkprogramreferstoa
programwrittenbyyouintheawkprogramminglanguage.
Primarily,thisWebpageexplainsthefeaturesofawkasdefinedinthePOSIXstandard.Itdoessointhecontextofthe
gawkimplementation.Whiledoingso,italsoattemptstodescribeimportantdifferencesbetweengawkandotherawk
implementations.5Finally,itnotesanygawkfeaturesthatarenotinthePOSIXstandardforawk.
ThisWebpagehasthedifficulttaskofbeingbothatutorialandareference.Ifyouareanovice,feelfreetoskipover
detailsthatseemtoocomplex.Youshouldalsoignorethemanycrossreferencestheyarefortheexpertuserandforthe
InfoandHTMLversionsoftheWebpage.
21/479
29/7/2016
TheGNUAwkUsersGuide
TherearesidebarsscatteredthroughouttheWebpage.Theyaddamorecompleteexplanationofpointsthatarerelevant,
butnotlikelytobeofinterestonfirstreading.Allappearintheindex,undertheheadingsidebar.
Mostofthetime,theexamplesusecompleteawkprograms.Someofthemoreadvancedsectionsshowonlythepartof
theawkprogramthatillustratestheconceptbeingdescribed.
AlthoughthisWebpageisaimedprincipallyatpeoplewhohavenotbeenexposedtoawk,thereisalotofinformation
herethateventheawkexpertshouldfinduseful.Inparticular,thedescriptionofPOSIXawkandtheexampleprograms
inLibraryFunctions,andinSamplePrograms,shouldbeofinterest.
ThisWebpageissplitintoseveralparts,asfollows:
PartIdescribestheawklanguageandthegawkprogramindetail.Itstartswiththebasics,andcontinuesthroughall
ofthefeaturesofawk.Itcontainsthefollowingchapters:
GettingStarted,providestheessentialsyouneedtoknowtobeginusingawk.
InvokingGawk,describeshowtorungawk,themeaningofitscommandlineoptions,andhowitfindsawk
programsourcefiles.
Regexp,introducesregularexpressionsingeneral,andinparticulartheflavorssupportedbyPOSIXawk
andgawk.
ReadingFiles,describeshowawkreadsyourdata.Itintroducestheconceptsofrecordsandfields,aswell
asthegetlinecommand.I/Oredirectionisfirstdescribedhere.NetworkI/Oisalsobrieflyintroducedhere.
Printing,describeshowawkprogramscanproduceoutputwithprintandprintf.
Expressions,describesexpressions,whicharethebasicbuildingblocksforgettingmostthingsdoneina
program.
PatternsandActions,describeshowtowritepatternsformatchingrecords,actionsfordoingsomething
whenarecordismatched,andthepredefinedvariablesawkandgawkuse.
Arrays,coversawksoneandonlydatastructure:theassociativearray.Deletingarrayelementsandwhole
arraysisdescribed,aswellassortingarraysingawk.Thechapteralsodescribeshowgawkprovidesarraysof
arrays.
Functions,describesthebuiltinfunctionsawkandgawkprovide,aswellashowtodefineyourown
functions.Italsodiscusseshowgawkletsyoucallfunctionsindirectly.
PartIIshowshowtouseawkandgawkforproblemsolving.Thereislotsofcodehereforyoutoreadandlearn
from.Thispartcontainsthefollowingchapters:
LibraryFunctions,providesanumberoffunctionsmeanttobeusedfrommainawkprograms.
SamplePrograms,providesmanysampleawkprograms.
Readingthesetwochaptersallowsyoutoseeawksolvingrealproblems.
PartIIIfocusesonfeaturesspecifictogawk.Itcontainsthefollowingchapters:
AdvancedFeatures,describesanumberofadvancedfeatures.Ofparticularnotearetheabilitiestocontrol
theorderofarraytraversal,havetwowaycommunicationswithanotherprocess,performTCP/IP
networking,andprofileyourawkprograms.
Internationalization,describesspecialfeaturesfortranslatingprogrammessagesintodifferentlanguages
atruntime.
Debugger,describesthegawkdebugger.
ArbitraryPrecisionArithmetic,describesadvancedarithmeticfacilities.
DynamicExtensions,describeshowtoaddnewvariablesandfunctionstogawkbywritingextensionsinC
orC++.
PartIVprovidestheappendices,theGlossary,andtwolicensesthatcoverthegawksourcecodeandthisWeb
page,respectively.Itcontainsthefollowingappendices:
LanguageHistory,describeshowtheawklanguagehasevolvedsinceitsfirstreleasetothepresent.Italso
describeshowgawkhasacquiredfeaturesovertime.
Installation,describeshowtogetgawk,howtocompileitonPOSIXcompatiblesystems,andhowto
compileanduseitondifferentnonPOSIXsystems.Italsodescribeshowtoreportbugsingawkandwhere
togetotherfreelyavailableawkimplementations.
Notes,describeshowtodisablegawksextensions,aswellashowtocontributenewcodetogawk,and
somepossiblefuturedirectionsforgawkdevelopment.
BasicConcepts,providessomeverycursorybackgroundmaterialforthosewhoarecompletelyunfamiliar
22/479
29/7/2016
TheGNUAwkUsersGuide
withcomputerprogramming.
TheGlossary,definesmost,ifnotall,ofthesignificanttermsusedthroughouttheWebpage.Ifyoufind
termsthatyouarentfamiliarwith,trylookingthemuphere.
Copying,andGNUFreeDocumentationLicense,presentthelicensesthatcoverthegawksourcecodeand
thisWebpage,respectively.
Next:ManualHistory,Previous:ThisManual,Up:Preface[Contents][Index]
TypographicalConventions
ThisWebpageiswritteninTexinfo,theGNUdocumentationformattinglanguage.AsingleTexinfosourcefileisused
toproduceboththeprintedandonlineversionsofthedocumentation.Becauseofthis,thetypographicalconventionsare
slightlydifferentthaninotherbooksyoumayhaveread.
Examplesyouwouldtypeatthecommandlineareprecededbythecommonshellprimaryandsecondaryprompts,$
and>.Inputthatyoutypeisshownlikethis.Outputfromthecommandisprecededbytheglyph|.Thistypically
representsthecommandsstandardoutput.Errormessagesandotheroutputonthecommandsstandarderrorare
precededbytheglypherror.Forexample:
$echohionstdout
|hionstdout
$echohelloonstderr1>&2
errorhelloonstderr
Inthetext,almostanythingrelatedtoprogramming,suchascommandnames,variableandfunctionnames,andstring,
numericandregexpconstantsappearinthisfont.Codefragmentsappearinthesamefontandquoted,likethis.
Thingsthatarereplacedbytheuserorprogrammerappearinthisfont.Optionslooklikethis:f.Filenamesare
indicatedlikethis:/path/to/ourfile.Somethingsareemphasizedlikethis,andifapointneedstobemadestrongly,it
isdonelikethis.Thefirstoccurrenceofanewtermisusuallyitsdefinitionandappearsinthesamefontastheprevious
occurrenceofdefinitioninthissentence.
Charactersthatyoutypeatthekeyboardlooklikethis.Inparticular,therearespecialcharacterscalledcontrol
characters.ThesearecharactersthatyoutypebyholdingdownboththeCONTROLkeyandanotherkey,atthesametime.
Forexample,aCtrldistypedbyfirstpressingandholdingtheCONTROLkey,nextpressingthedkey,andfinally
releasingbothkeys.
Forthesakeofbrevity,throughoutthisWebpage,werefertoBrianKernighansversionofawkasBWKawk.(See
OtherVersions,forinformationonhisandotherversions.)
DarkCorners
Darkcornersarebasicallyfractalnomatterhowmuchyouilluminate,theresalwaysasmallerbut
darkerone.
BrianKernighan
UntilthePOSIXstandard(andGAWK:EffectiveAWKProgramming),manyfeaturesofawkwereeitherpoorly
documentedornotdocumentedatall.Descriptionsofsuchfeatures(oftencalleddarkcorners)arenotedinthisWeb
pagewith(d.c.).Theyalsoappearintheindexundertheheadingdarkcorner.
But,asnotedbytheopeningquote,anycoverageofdarkcornersisbydefinitionincomplete.
Extensionstothestandardawklanguagethataresupportedbymorethanoneawkimplementationaremarked(c.e.),
andlistedintheindexundercommonextensionsandextensions,common.
23/479
29/7/2016
TheGNUAwkUsersGuide
Next:HowToContribute,Previous:Conventions,Up:Preface[Contents][Index]
TheGNUProjectandThisBook
TheFreeSoftwareFoundation(FSF)isanonprofitorganizationdedicatedtotheproductionanddistributionoffreely
distributablesoftware.ItwasfoundedbyRichardM.Stallman,theauthoroftheoriginalEmacseditor.GNUEmacsis
themostwidelyusedversionofEmacstoday.
TheGNU6ProjectisanongoingeffortonthepartoftheFreeSoftwareFoundationtocreateacomplete,freely
distributable,POSIXcompliantcomputingenvironment.TheFSFusestheGNUGeneralPublicLicense(GPL)to
ensurethatitssoftwaressourcecodeisalwaysavailabletotheenduser.AcopyoftheGPLisincludedinthisWeb
pageforyourreference(seeCopying).TheGPLappliestotheClanguagesourcecodeforgawk.Tofindoutmoreabout
theFSFandtheGNUProjectonline,seetheGNUProjectshomepage.ThisWebpagemayalsobereadfromGNUs
website.
Ashell,aneditor(Emacs),highlyportableoptimizingC,C++,andObjectiveCcompilers,asymbolicdebuggerand
dozensoflargeandsmallutilities(suchasgawk),haveallbeencompletedandarefreelyavailable.TheGNUoperating
systemkernel(theHURD),hasbeenreleasedbutremainsinanearlystageofdevelopment.
UntiltheGNUoperatingsystemismorefullydeveloped,youshouldconsiderusingGNU/Linux,afreelydistributable,
UnixlikeoperatingsystemforIntel,PowerArchitecture,SunSPARC,IBMS/390,andothersystems.7Many
GNU/LinuxdistributionsareavailablefordownloadfromtheInternet.
TheWebpageyouarereadingisactuallyfreeatleast,theinformationinitisfreetoanyone.Themachinereadable
sourcecodefortheWebpagecomeswithgawk.(TakeamomenttochecktheFreeDocumentationLicenseinGNUFree
DocumentationLicense.)
TheWebpageitselfhasgonethroughmultiplepreviouseditions.PaulRubinwrotetheveryfirstdraftofTheGAWK
Manualitwasaround40pageslong.DianeCloseandRichardStallmanimprovedit,yieldingaversionthatwas
around90pagesandbarelydescribedtheoriginal,oldversionofawk.
Istartedworkingwiththatversioninthefallof1988.Asworkonitprogressed,theFSFpublishedseveralpreliminary
versions(numbered0.x).In1996,edition1.0wasreleasedwithgawk3.0.0.TheFSFpublishedthefirsttwoeditions
underthetitleTheGNUAwkUsersGuide.
Thiseditionmaintainsthebasicstructureofthepreviouseditions.ForFSFedition4.0,thecontentwasthoroughly
reviewedandupdated.Allreferencestogawkversionspriorto4.0wereremoved.Ofsignificantnoteforthateditionwas
theadditionofDebugger.
ForFSFedition4.1,thecontenthasbeenreorganizedintoparts,andthemajornewadditionsareArbitraryPrecision
Arithmetic,andDynamicExtensions.
ThisWebpagewillundoubtedlycontinuetoevolve.IfyoufindanerrorintheWebpage,pleasereportit!SeeBugs,for
informationonsubmittingproblemreportselectronically.
Next:Acknowledgments,Previous:ManualHistory,Up:Preface[Contents][Index]
HowtoContribute
AsthemaintainerofGNUawk,IoncethoughtthatIwouldbeabletomanageacollectionofpubliclyavailableawk
programsandIevensolicitedcontributions.MakingthingsavailableontheInternethelpskeepthegawkdistribution
downtomanageablesize.
Theinitialcollectionofmaterial,suchasitis,isstillavailableatftp://ftp.freefriends.org/arnold/Awkstuff.Inthehopes
ofdoingsomethingmorebroad,Iacquiredtheawk.infodomain.
However,IfoundthatIcouldnotdedicateenoughtimetomanagingcontributedcode:thearchivedidnotgrowandthe
24/479
29/7/2016
TheGNUAwkUsersGuide
domainwentunusedforseveralyears.
Latein2008,avolunteertookonthetaskofsettingupanawkrelatedwebsitehttp://awk.infoanddidaverynice
job.
Ifyouhavewrittenaninterestingawkprogram,orhavewrittenagawkextensionthatyouwouldliketosharewiththe
restoftheworld,pleaseseehttp://awk.info/?contributeforhowtocontributeittothewebsite.
Previous:HowToContribute,Up:Preface[Contents][Index]
Acknowledgments
TheinitialdraftofTheGAWKManualhadthefollowingacknowledgments:
Manypeopleneedtobethankedfortheirassistanceinproducingthismanual.JayFenlasoncontributed
manyideasandsampleprograms.RichardMlynarikandRobertChassellgavehelpfulcommentsondrafts
ofthismanual.ThepaperASupplementalDocumentforAWKbyJohnW.PierceoftheChemistry
DepartmentatUCSanDiego,pinpointedseveralissuesrelevantbothtoawkimplementationandtothis
manual,thatwouldotherwisehaveescapedus.
IwouldliketoacknowledgeRichardM.Stallman,forhisvisionofabetterworldandforhiscourageinfoundingthe
FSFandstartingtheGNUProject.
EarliereditionsofthisWebpagehadthefollowingacknowledgements:
Thefollowingpeople(inalphabeticalorder)providedhelpfulcommentsonvariousversionsofthisbook:
RickAdams,Dr.NelsonH.F.Beebe,KarlBerry,Dr.MichaelBrennan,RichBurridge,ClaireCloutier,
DianeClose,ScottDeifik,Christopher(Topher)Eliot,JeffreyFriedl,Dr.DarrelHankerson,Michal
Jaegermann,Dr.RichardJ.LeBlanc,MichaelLijewski,PatRankin,MiriamRobbins,MarySheehan,and
ChuckToporek.
RobertJ.ChassellprovidedmuchvaluableadviceontheuseofTexinfo.Healsodeservesspecialthanks
forconvincingmenottotitlethisWebpageHowtoGawkPolitely.KarlBerryhelpedsignificantlywiththe
TeXpartofTexinfo.
IwouldliketothankMarshallandElaineHartholzofSeattleandDr.BertandRitaSchreiberofDetroitfor
largeamountsofquietvacationtimeintheirhomes,whichallowedmetomakesignificantprogressonthis
Webpageandongawkitself.
PhilHughesofSSCcontributedinaveryimportantwaybyloaningmehislaptopGNU/Linuxsystem,not
once,buttwice,whichallowedmetodoalotofworkwhileawayfromhome.
DavidTruemandeservesspecialcredithehasdoneayeomanjobofevolvinggawksothatitperformswell
andwithoutbugs.Althoughheisnolongerinvolvedwithgawk,workingwithhimonthisprojectwasa
significantpleasure.
TheintrepidmembersoftheGNITSmailinglist,andmostnotablyUlrichDrepper,providedinvaluable
helpandfeedbackforthedesignoftheinternationalizationfeatures.
ChuckToporek,MarySheehan,andClaireCloutierofOReilly&Associatescontributedsignificant
editorialhelpforthisWebpageforthe3.1releaseofgawk.
Dr.NelsonBeebe,AndreasBuening,Dr.ManuelCollado,AntonioColombo,StephenDavies,ScottDeifik,Akim
Demaille,DarrelHankerson,MichalJaegermann,JrgenKahrs,StepanKasal,JohnMalmberg,DavePitts,Chet
Ramey,PatRankin,AndrewSchorr,CorinnaVinschen,andEliZaretskii(inalphabeticalorder)makeupthecurrent
gawkcrackportabilityteam.Withouttheirhardworkandhelp,gawkwouldnotbenearlytherobust,portableprogram
itistoday.Ithasbeenandcontinuestobeapleasureworkingwiththisteamoffinepeople.
25/479
29/7/2016
TheGNUAwkUsersGuide
Notablecodeanddocumentationcontributionsweremadebyanumberofpeople.SeeContributors,forthefulllist.
ThankstoMichaelBrennanfortheForewords.
ThankstoPatriceDumasforthenewmakeinfoprogram.ThankstoKarlBerry,whocontinuestoworktokeepthe
Texinfomarkuplanguagesane.
RobertP.J.Day,MichaelBrennan,andBrianKernighankindlyactedasreviewersforthe2015editionofthisWebpage.
Theirfeedbackhelpedimprovethefinalwork.
IwouldalsoliketothankBrianKernighanforhisinvaluableassistanceduringthetestinganddebuggingofgawk,and
forhisongoinghelpandadviceinclarifyingnumerouspointsaboutthelanguage.Wecouldnothavedonenearlyas
goodajoboneithergawkoritsdocumentationwithouthishelp.
Brianisinaclassbyhimselfasaprogrammerandtechnicalauthor.Ihavetothankhim(yetagain)forhisongoing
friendshipandforbeingarolemodeltomeforcloseto30years!Havinghimasareviewerisanexcitingprivilege.It
hasalsobeenextremelyhumbling...
Imustthankmywonderfulwife,Miriam,forherpatiencethroughthemanyversionsofthisproject,forher
proofreading,andforsharingmewiththecomputer.Iwouldliketothankmyparentsfortheirlove,andforthegrace
withwhichtheyraisedandeducatedme.Finally,IalsomustacknowledgemygratitudetoGd,forthemany
opportunitiesHehassentmyway,aswellasforthegiftsHehasgivenmewithwhichtotakeadvantageofthose
opportunities.
ArnoldRobbins
NofAyalon
Israel
February2015
Next:InvokingGawk,Previous:Preface,Up:Top[Contents][Index]
PartI:
TheawkLanguage
Thebasicfunctionofawkistosearchfilesforlines(orotherunitsoftext)thatcontaincertainpatterns.Whenaline
matchesoneofthepatterns,awkperformsspecifiedactionsonthatline.awkcontinuestoprocessinputlinesinthisway
untilitreachestheendoftheinputfiles.
Programsinawkaredifferentfromprogramsinmostotherlanguages,becauseawkprogramsaredatadriven(i.e.,you
describethedatayouwanttoworkwithandthenwhattodowhenyoufindit).Mostotherlanguagesareprocedural
youhavetodescribe,ingreatdetail,everysteptheprogramshouldtake.Whenworkingwithprocedurallanguages,itis
usuallymuchhardertoclearlydescribethedatayourprogramwillprocess.Forthisreason,awkprogramsareoften
refreshinglyeasytoreadandwrite.
Whenyourunawk,youspecifyanawkprogramthattellsawkwhattodo.Theprogramconsistsofaseriesofrules(it
mayalsocontainfunctiondefinitions,anadvancedfeaturethatwewillignorefornowseeUserdefined).Eachrule
specifiesonepatterntosearchforandoneactiontoperformuponfindingthepattern.
26/479
29/7/2016
TheGNUAwkUsersGuide
Syntactically,aruleconsistsofapatternfollowedbyanaction.Theactionisenclosedinbracestoseparateitfromthe
pattern.Newlinesusuallyseparaterules.Therefore,anawkprogramlookslikethis:
pattern{action}
pattern{action}
Runninggawk:
Howtorungawkprogramsincludescommandlinesyntax.
SampleDataFiles: SampledatafilesforuseintheawkprogramsillustratedinthisWebpage.
VerySimple:
Averysimpleexample.
TwoRules:
Alesssimpleonelineexampleusingtworules.
MoreComplex: Amorecomplexexample.
Statements/Lines: Subdividingorcombiningstatementsintolines.
OtherFeatures:
OtherFeaturesofawk.
When:
Whentousegawkandwhentouseotherthings.
IntroSummary: Summaryoftheintroduction.
Next:SampleDataFiles,Up:GettingStarted[Contents][Index]
1.1HowtoRunawkPrograms
Thereareseveralwaystorunanawkprogram.Iftheprogramisshort,itiseasiesttoincludeitinthecommandthatruns
awk,likethis:
awk'program'inputfile1inputfile2
Whentheprogramislong,itisusuallymoreconvenienttoputitinafileandrunitwithacommandlikethis:
awkfprogramfileinputfile1inputfile2
Thissectiondiscussesbothmechanisms,alongwithseveralvariationsofeach.
Oneshot:
Runningashortthrowawayawkprogram.
ReadTerminal:
Usingnoinputfiles(inputfromthekeyboardinstead).
Long:
Puttingpermanentawkprogramsinfiles.
ExecutableScripts: Makingselfcontainedawkprograms.
Comments:
Addingdocumentationtogawkprograms.
Quoting:
Morediscussionofshellquotingissues.
Next:ReadTerminal,Up:Runninggawk[Contents][Index]
1.1.1OneShotThrowawayawkPrograms
Onceyouarefamiliarwithawk,youwilloftentypeinsimpleprogramsthemomentyouwanttousethem.Thenyoucan
writetheprogramasthefirstargumentoftheawkcommand,likethis:
awk'program'inputfile1inputfile2
whereprogramconsistsofaseriesofpatternsandactions,asdescribedearlier.
Thiscommandformatinstructstheshell,orcommandinterpreter,tostartawkandusetheprogramtoprocessrecordsin
theinputfile(s).Therearesinglequotesaroundprogramsotheshellwontinterpretanyawkcharactersasspecialshell
characters.Thequotesalsocausetheshelltotreatallofprogramasasingleargumentforawk,andallowprogramtobe
morethanonelinelong.
Thisformatisalsousefulforrunningshortormediumsizedawkprogramsfromshellscripts,becauseitavoidstheneed
foraseparatefilefortheawkprogram.Aselfcontainedshellscriptismorereliablebecausetherearenootherfilesto
27/479
29/7/2016
TheGNUAwkUsersGuide
misplace.
Laterinthischapter,inVerySimple,wellseeexamplesofseveralshort,selfcontainedprograms.
Next:Long,Previous:Oneshot,Up:Runninggawk[Contents][Index]
1.1.2RunningawkWithoutInputFiles
Youcanalsorunawkwithoutanyinputfiles.Ifyoutypethefollowingcommandline:
awk'program'
awkappliestheprogramtothestandardinput,whichusuallymeanswhateveryoutypeonthekeyboard.Thiscontinues
untilyouindicateendoffilebytypingCtrld.(OnnonPOSIXoperatingsystems,theendoffilecharactermaybe
different.Forexample,onOS/2,itisCtrlz.)
Asanexample,thefollowingprogramprintsafriendlypieceofadvice(fromDouglasAdamssTheHitchhikersGuide
totheGalaxy),tokeepyoufromworryingaboutthecomplexitiesofcomputerprogramming:
$awk'BEGIN{print"Don\47tPanic!"}'
|Don'tPanic!
awkexecutesstatementsassociatedwithBEGINbeforereadinganyinput.Iftherearenootherstatementsinyour
program,asisthecasehere,awkjuststops,insteadoftryingtoreadinputitdoesntknowhowtoprocess.The\47isa
magicway(explainedlater)ofgettingasinglequoteintotheprogram,withouthavingtoengageinuglyshellquoting
tricks.
NOTE:IfyouuseBashasyourshell,youshouldexecutethecommandset+Hbeforerunningthis
programinteractively,todisabletheCshellstylecommandhistory,whichtreats!asaspecialcharacter.
Werecommendputtingthiscommandintoyourpersonalstartupfile.
Thisnextsimpleawkprogramemulatesthecatutilityitcopieswhateveryoutypeonthekeyboardtoitsstandard
output(whythisworksisexplainedshortly):
$awk'{print}'
Nowisthetimeforallgoodmen
|Nowisthetimeforallgoodmen
tocometotheaidoftheircountry.
|tocometotheaidoftheircountry.
Fourscoreandsevenyearsago,...
|Fourscoreandsevenyearsago,...
What,meworry?
|What,meworry?
Ctrld
Next:ExecutableScripts,Previous:ReadTerminal,Up:Runninggawk[Contents][Index]
1.1.3RunningLongPrograms
Sometimesawkprogramsareverylong.Inthesecases,itismoreconvenienttoputtheprogramintoaseparatefile.In
ordertotellawktousethatfileforitsprogram,youtype:
awkfsourcefileinputfile1inputfile2
Thefinstructstheawkutilitytogettheawkprogramfromthefilesourcefile(seeOptions).Anyfilenamecanbeused
forsourcefile.Forexample,youcouldputtheprogram:
BEGIN{print"Don'tPanic!"}
28/479
29/7/2016
TheGNUAwkUsersGuide
intothefileadvice.Thenthiscommand:
awkfadvice
doesthesamethingasthisone:
awk'BEGIN{print"Don\47tPanic!"}'
Thiswasexplainedearlier(seeReadTerminal).Notethatyoudontusuallyneedsinglequotesaroundthefilenamethat
youspecifywithf,becausemostfilenamesdontcontainanyoftheshellsspecialcharacters.Noticethatinadvice,
theawkprogramdidnothavesinglequotesaroundit.Thequotesareonlyneededforprogramsthatareprovidedonthe
awkcommandline.(Also,placingtheprograminafileallowsustousealiteralsinglequoteintheprogramtext,instead
ofthemagic\47.)
Ifyouwanttoclearlyidentifyanawkprogramfileassuch,youcanaddtheextension.awktothefilename.Thisdoesnt
affecttheexecutionoftheawkprogrambutitdoesmakehousekeepingeasier.
Next:Comments,Previous:Long,Up:Runninggawk[Contents][Index]
1.1.4ExecutableawkPrograms
Onceyouhavelearnedawk,youmaywanttowriteselfcontainedawkscripts,usingthe#!scriptmechanism.Youcan
dothisonmanysystems.8Forexample,youcouldupdatethefileadvicetolooklikethis:
#!/bin/awkf
Aftermakingthisfileexecutable(withthechmodutility),simplytypeadviceattheshellandthesystemarrangesto
runawkasifyouhadtypedawkfadvice:
$chmod+xadvice
$advice
|Don'tPanic!
(Weassumeyouhavethecurrentdirectoryinyourshellssearchpathvariable[typically$PATH].Ifnot,youmayneedto
type./adviceattheshell.)
Selfcontainedawkscriptsareusefulwhenyouwanttowriteaprogramthatuserscaninvokewithouttheirhavingto
knowthattheprogramiswritteninawk.
Understanding#!
awkisaninterpretedlanguage.Thismeansthattheawkutilityreadsyourprogramandthenprocessesyourdata
accordingtotheinstructionsinyourprogram.(ThisisdifferentfromacompiledlanguagesuchasC,whereyour
programisfirstcompiledintomachinecodethatisexecuteddirectlybyyoursystemsprocessor.)Theawkutilityis
thustermedaninterpreter.Manymodernlanguagesareinterpreted.
Thelinebeginningwith#!liststhefullfilenameofaninterpretertorunandasingleoptionalinitialcommandline
argumenttopasstothatinterpreter.Theoperatingsystemthenrunstheinterpreterwiththegivenargumentandthefull
argumentlistoftheexecutedprogram.Thefirstargumentinthelististhefullfilenameoftheawkprogram.Therest
oftheargumentlistcontainseitheroptionstoawk,ordatafiles,orboth.(Notethatonmanysystemsawkmaybefound
in/usr/bininsteadofin/bin.)
Somesystemslimitthelengthoftheinterpreternameto32characters.Often,thiscanbedealtwithbyusinga
symboliclink.
Youshouldnotputmorethanoneargumentonthe#!lineafterthepathto awk.Itdoesnotwork.Theoperating
systemtreatstherestofthelineasasingleargumentandpassesittoawk.Doingthisleadstoconfusingbehavior
29/479
29/7/2016
TheGNUAwkUsersGuide
mostlikelyausagediagnosticofsomesortfromawk.
Finally,thevalueofARGV[0](seeBuiltinVariables)variesdependinguponyouroperatingsystem.Somesystemsput
awkthere,someputthefullpathnameofawk(suchas/bin/awk),andsomeputthenameofyourscript(advice).
(d.c.)DontrelyonthevalueofARGV[0]toprovideyourscriptname.
Next:Quoting,Previous:ExecutableScripts,Up:Runninggawk[Contents][Index]
1.1.5CommentsinawkPrograms
Acommentissometextthatisincludedinaprogramforthesakeofhumanreadersitisnotreallyanexecutablepartof
theprogram.Commentscanexplainwhattheprogramdoesandhowitworks.Nearlyallprogramminglanguageshave
provisionsforcomments,asprogramsaretypicallyhardtounderstandwithoutthem.
Intheawklanguage,acommentstartswiththenumbersigncharacter(#)andcontinuestotheendoftheline.The#
doesnothavetobethefirstcharacterontheline.Theawklanguageignorestherestofalinefollowinganumbersign.
Forexample,wecouldhaveputthefollowingintoadvice:
#Thisprogramprintsanice,friendlymessage.Ithelps
#keepnoviceusersfrombeingafraidofthecomputer.
Youcanputcommentlinesintokeyboardcomposedthrowawayawkprograms,butthisusuallyisntveryusefulthe
purposeofacommentistohelpyouoranotherpersonunderstandtheprogramwhenreadingitatalatertime.
CAUTION:AsmentionedinOneshot,youcanencloseshorttomediumsizedprogramsinsinglequotes,
inordertokeepyourshellscriptsselfcontained.Whendoingso,dontputanapostrophe(i.e.,asingle
quote)intoacomment(oranywhereelseinyourprogram).Theshellinterpretsthequoteastheclosing
quotefortheentireprogram.Asaresult,usuallytheshellprintsamessageaboutmismatchedquotes,andif
awkactuallyruns,itwillprobablyprintstrangemessagesaboutsyntaxerrors.Forexample,lookatthe
following:
$awk'BEGIN{print"hello"}#let'sbecute'
>
Theshellseesthatthefirsttwoquotesmatch,andthatanewquotedobjectbeginsattheendofthe
commandline.Itthereforepromptswiththesecondaryprompt,waitingformoreinput.WithUnixawk,
closingthequotedstringproducesthisresult:
$awk'{print"hello"}#let'sbecute'
>'
errorawk:can'topenfilebe
errorsourcelinenumber1
Puttingabackslashbeforethesinglequoteinlet'swouldnthelp,becausebackslashesarenotspecial
insidesinglequotes.Thenextsubsectiondescribestheshellsquotingrules.
Previous:Comments,Up:Runninggawk[Contents][Index]
1.1.6ShellQuotingIssues
DOSQuoting: QuotinginWindowsBatchFiles.
Forshorttomediumlengthawkprograms,itismostconvenienttoentertheprogramontheawkcommandline.Thisis
bestdonebyenclosingtheentireprograminsinglequotes.Thisistruewhetheryouareenteringtheprogram
interactivelyattheshellprompt,orwritingitaspartofalargershellscript:
30/479
29/7/2016
TheGNUAwkUsersGuide
awk'programtext'inputfile1inputfile2
Onceyouareworkingwiththeshell,itishelpfultohaveabasicknowledgeofshellquotingrules.Thefollowingrules
applyonlytoPOSIXcompliant,Bournestyleshells(suchasBash,theGNUBourneAgainShell).IfyouusetheC
shell,youreonyourown.
Beforedivingintotherules,weintroduceaconceptthatappearsthroughoutthisWebpage,whichisthatofthenull,or
empty,string.
Thenullstringischaracterdatathathasnovalue.Inotherwords,itisempty.Itiswritteninawkprogramslikethis:"".
Intheshell,itcanbewrittenusingsingleordoublequotes:""or''.Althoughthenullstringhasnocharactersinit,it
doesexist.Forexample,considerthiscommand:
$echo""
Here,theechoutilityreceivesasingleargument,eventhoughthatargumenthasnocharactersinit.Intherestofthis
Webpage,weusethetermsnullstringandemptystringinterchangeably.Now,ontothequotingrules:
Quoteditemscanbeconcatenatedwithnonquoteditemsaswellaswithotherquoteditems.Theshellturns
everythingintooneargumentforthecommand.
Precedinganysinglecharacterwithabackslash(\)quotesthatcharacter.Theshellremovesthebackslashand
passesthequotedcharacterontothecommand.
Singlequotesprotecteverythingbetweentheopeningandclosingquotes.Theshelldoesnointerpretationofthe
quotedtext,passingitonverbatimtothecommand.Itisimpossibletoembedasinglequoteinsidesinglequoted
text.ReferbacktoComments,foranexampleofwhathappensifyoutry.
Doublequotesprotectmostthingsbetweentheopeningandclosingquotes.Theshelldoesatleastvariableand
commandsubstitutiononthequotedtext.Differentshellsmaydoadditionalkindsofprocessingondouble
quotedtext.
Becausecertaincharacterswithindoublequotedtextareprocessedbytheshell,theymustbeescapedwithinthe
text.Ofnotearethecharacters$,`,\,and",allofwhichmustbeprecededbyabackslashwithindouble
quotedtextiftheyaretobepassedonliterallytotheprogram.(Theleadingbackslashisstrippedfirst.)Thus,the
exampleseenpreviouslyinReadTerminal:
awk'BEGIN{print"Don\47tPanic!"}'
couldinsteadbewrittenthisway:
$awk"BEGIN{print\"Don'tPanic!\"}"
|Don'tPanic!
Notethatthesinglequoteisnotspecialwithindoublequotes.
Nullstringsareremovedwhentheyoccuraspartofanonnullcommandlineargument,whileexplicitnull
objectsarekept.Forexample,tospecifythatthefieldseparatorFSshouldbesettothenullstring,use:
awkF""'program'files#correct
Dontusethis:
awkF""'program'files#wrong!
Inthesecondcase,awkattemptstousethetextoftheprogramasthevalueofFS,andthefirstfilenameasthetext
oftheprogram!Thisresultsinsyntaxerrorsatbest,andconfusingbehavioratworst.
Mixingsingleanddoublequotesisdifficult.Youhavetoresorttoshellquotingtricks,likethis:
$awk'BEGIN{print"Hereisasinglequote<'"'"'>"}'
|Hereisasinglequote<'>
Thisprogramconsistsofthreeconcatenatedquotedstrings.Thefirstandthethirdaresinglequoted,andthesecondis
31/479
29/7/2016
TheGNUAwkUsersGuide
doublequoted.
Thiscanbesimplifiedto:
$awk'BEGIN{print"Hereisasinglequote<'\''>"}'
Judgeforyourselfwhichofthesetwoisthemorereadable.
Anotheroptionistousedoublequotes,escapingtheembedded,awkleveldoublequotes:
$awk"BEGIN{print\"Hereisasinglequote<'>\"}"
Thisoptionisalsopainful,becausedoublequotes,backslashes,anddollarsignsareverycommoninmoreadvancedawk
programs.
Athirdoptionistousetheoctalescapesequenceequivalents(seeEscapeSequences)forthesingleanddoublequote
characters,likeso:
$awk'BEGIN{print"Hereisasinglequote<\47>"}'
$awk'BEGIN{print"Hereisadoublequote<\42>"}'
|Hereisadoublequote<">
Thisworksnicely,butyoushouldcommentclearlywhattheescapesmean.
Afourthoptionistousecommandlinevariableassignment,likethis:
$awkvsq="'"'BEGIN{print"Hereisasinglequote<"sq">"}'
(Here,thetwostringconstantsandthevalueofsqareconcatenatedintoasinglestringthatisprintedbyprint.)
Ifyoureallyneedbothsingleanddoublequotesinyourawkprogram,itisprobablybesttomoveitintoaseparatefile,
wheretheshellwontbepartofthepictureandyoucansaywhatyoumean.
Up:Quoting[Contents][Index]
1.1.6.1QuotinginMSWindowsBatchFiles
AlthoughthisWebpagegenerallyonlyworriesaboutPOSIXsystemsandthePOSIXshell,thefollowingissuearises
oftenenoughformanyusersthatitisworthaddressing.
TheshellsonMicrosoftWindowssystemsusethedoublequotecharacterforquoting,andmakeitdifficultor
impossibletoincludeanescapeddoublequotecharacterinacommandlinescript.Thefollowingexample,courtesyof
JeroenBrink,showshowtoprintalllinesinafilesurroundedbydoublequotes:
gawk"{print\"\042\"$0\"\042\"}"file
Next:VerySimple,Previous:Runninggawk,Up:GettingStarted[Contents][Index]
1.2DatafilesfortheExamples
ManyoftheexamplesinthisWebpagetaketheirinputfromtwosampledatafiles.Thefirst,maillist,representsalist
ofpeoplesnamestogetherwiththeiremailaddressesandinformationaboutthosepeople.Theseconddatafile,called
inventoryshipped,containsinformationaboutmonthlyshipments.Inbothfiles,eachlineisconsideredtobeone
record.
32/479
29/7/2016
TheGNUAwkUsersGuide
Inmaillist,eachrecordcontainsthenameofaperson,his/herphonenumber,his/heremailaddress,andacodefor
his/herrelationshipwiththeauthorofthelist.Thecolumnsarealignedusingspaces.AnAinthelastcolumnmeans
thatthepersonisanacquaintance.AnFinthelastcolumnmeansthatthepersonisafriend.AnRmeansthatthe
personisarelative:
Amelia5555553amelia.zodiacusque@gmail.comF
Anthony5553412anthony.asserturo@hotmail.comA
Becky5557685becky.algebrarum@gmail.comA
Bill5551675bill.drowning@hotmail.comA
Broderick5550542broderick.aliquotiens@yahoo.comR
Camilla5552912camilla.infusarum@skynet.beR
Fabius5551234fabius.undevicesimus@ucb.eduF
Julie5556699julie.perscrutabor@skeeve.comF
Martin5556480martin.codicibus@hotmail.comA
Samuel5553430samuel.lanceolis@shu.eduA
JeanPaul5552127jeanpaul.campanorum@nyu.eduR
Thedatafileinventoryshippedrepresentsinformationaboutshipmentsduringtheyear.Eachrecordcontainsthe
month,thenumberofgreencratesshipped,thenumberofredboxesshipped,thenumberoforangebagsshipped,and
thenumberofbluepackagesshipped,respectively.Thereare16entries,coveringthe12monthsoflastyearandthefirst
fourmonthsofthecurrentyear.Anemptylineseparatesthedataforthetwoyears:
Jan132515115
Feb153224226
Mar152434228
Apr315263420
May163429208
Jun314275492
Jul243467436
Aug153447316
Sep135537277
Oct295468525
Nov208782577
Dec173561401
Jan213664620
Feb265880652
Mar247570495
Apr217074514
Thesamplefilesareincludedinthegawkdistribution,inthedirectoryawklib/eg/data.
Next:TwoRules,Previous:SampleDataFiles,Up:GettingStarted[Contents][Index]
1.3SomeSimpleExamples
Thefollowingcommandrunsasimpleawkprogramthatsearchestheinputfilemaillistforthecharacterstringli(a
groupingofcharactersisusuallycalledastringthetermstringisbasedonsimilarusageinEnglish,suchasastringof
pearlsorastringofcarsinatrain):
awk'/li/{print$0}'maillist
Whenlinescontainingliarefound,theyareprintedbecauseprint$0meansprintthecurrentline.(Justprintby
itselfmeansthesamething,sowecouldhavewrittenthatinstead.)
Youwillnoticethatslashes(/)surroundthestringliinthe awkprogram.Theslashesindicatethatliisthepattern
tosearchfor.Thistypeofpatterniscalledaregularexpression,whichiscoveredinmoredetaillater(seeRegexp).The
patternisallowedtomatchpartsofwords.Therearesinglequotesaroundtheawkprogramsothattheshellwont
interpretanyofitasspecialshellcharacters.
Hereiswhatthisprogramprints:
33/479
29/7/2016
TheGNUAwkUsersGuide
$awk'/li/{print$0}'maillist
|Amelia5555553amelia.zodiacusque@gmail.comF
|Broderick5550542broderick.aliquotiens@yahoo.comR
|Julie5556699julie.perscrutabor@skeeve.comF
|Samuel5553430samuel.lanceolis@shu.eduA
Inanawkrule,eitherthepatternortheactioncanbeomitted,butnotboth.Ifthepatternisomitted,thentheactionis
performedforeveryinputline.Iftheactionisomitted,thedefaultactionistoprintalllinesthatmatchthepattern.
Thus,wecouldleaveouttheaction(theprintstatementandthebraces)inthepreviousexampleandtheresultwould
bethesame:awkprintsalllinesmatchingthepatternli.Bycomparison,omittingtheprintstatementbutretainingthe
bracesmakesanemptyactionthatdoesnothing(i.e.,nolinesareprinted).
Manypracticalawkprogramsarejustalineortwolong.Followingisacollectionofuseful,shortprogramstogetyou
started.Someoftheseprogramscontainconstructsthathaventbeencoveredyet.(Thedescriptionoftheprogramwill
giveyouagoodideaofwhatisgoingon,butyoullneedtoreadtherestoftheWebpagetobecomeanawkexpert!)
Mostoftheexamplesuseadatafilenameddata.Thisisjustaplaceholderifyouusetheseprogramsyourself,
substituteyourownfilenamesfordata.Forfuturereference,notethatthereisoftenmorethanonewaytodothingsin
awk.Atsomepoint,youmaywanttolookbackattheseexamplesandseeifyoucancomeupwithdifferentwaystodo
thesamethingsshownhere:
Printeverylinethatislongerthan80characters:
awk'length($0)>80'data
Thesolerulehasarelationalexpressionasitspatternandhasnoactionsoitusesthedefaultaction,printingthe
record.
Printthelengthofthelongestinputline:
awk'{if(length($0)>max)max=length($0)}
END{printmax}'data
ThecodeassociatedwithENDexecutesafterallinputhasbeenreaditstheothersideofthecointoBEGIN.
Printthelengthofthelongestlineindata:
expanddata|awk'{if(x<length($0))x=length($0)}
END{print"maximumlinelengthis"x}'
Thisexamplediffersslightlyfromthepreviousone:theinputisprocessedbytheexpandutilitytochangeTABs
intospaces,sothewidthscomparedareactuallytherightmargincolumns,asopposedtothenumberofinput
charactersoneachline.
Printeverylinethathasatleastonefield:
awk'NF>0'data
Thisisaneasywaytodeleteblanklinesfromafile(orrather,tocreateanewfilesimilartotheoldfilebutfrom
whichtheblanklineshavebeenremoved).
Printsevenrandomnumbersfrom0to100,inclusive:
awk'BEGIN{for(i=1;i<=7;i++)
printint(101*rand())}'
Printthetotalnumberofbytesusedbyfiles:
lslfiles|awk'{x+=$5}
END{print"totalbytes:"x}'
Printthetotalnumberofkilobytesusedbyfiles:
lslfiles|awk'{x+=$5}
34/479
29/7/2016
TheGNUAwkUsersGuide
END{print"totalKbytes:",x/1024}'
Printasortedlistoftheloginnamesofallusers:
awkF:'{print$1}'/etc/passwd|sort
Countthelinesinafile:
awk'END{printNR}'data
Printtheevennumberedlinesinthedatafile:
awk'NR%2==0'data
IfyouusedtheexpressionNR%2==1instead,theprogramwouldprinttheoddnumberedlines.
Next:MoreComplex,Previous:VerySimple,Up:GettingStarted[Contents][Index]
1.4AnExamplewithTwoRules
Theawkutilityreadstheinputfilesonelineatatime.Foreachline,awktriesthepatternsofeachrule.Ifseveralpatterns
match,thenseveralactionsexecuteintheorderinwhichtheyappearintheawkprogram.Ifnopatternsmatch,thenno
actionsrun.
Afterprocessingalltherulesthatmatchtheline(andperhapstherearenone),awkreadsthenextline.(However,see
NextStatement,andalsoseeNextfileStatement.)Thiscontinuesuntiltheprogramreachestheendofthefile.For
example,thefollowingawkprogramcontainstworules:
/12/{print$0}
/21/{print$0}
Thefirstrulehasthestring12asthepatternandprint$0astheaction.Thesecondrulehasthestring21asthe
patternandalsohasprint$0astheaction.Eachrulesactionisenclosedinitsownpairofbraces.
Thisprogramprintseverylinethatcontainsthestring12orthestring21.Ifalinecontainsbothstrings,itisprinted
twice,oncebyeachrule.
Thisiswhathappensifwerunthisprogramonourtwosampledatafiles,maillistandinventoryshipped:
$awk'/12/{print$0}
>/21/{print$0}'maillistinventoryshipped
|Anthony5553412anthony.asserturo@hotmail.comA
|Camilla5552912camilla.infusarum@skynet.beR
|Fabius5551234fabius.undevicesimus@ucb.eduF
|JeanPaul5552127jeanpaul.campanorum@nyu.eduR
|Jan213664620
|Apr217074514
NotehowthelinebeginningwithJeanPaulin maillistwasprintedtwice,onceforeachrule.
Next:Statements/Lines,Previous:TwoRules,Up:GettingStarted[Contents][Index]
1.5AMoreComplexExample
Nowthatwevemasteredsomesimpletasks,letslookatwhattypicalawkprogramsdo.Thisexampleshowshowawk
canbeusedtosummarize,select,andrearrangetheoutputofanotherutility.Itusesfeaturesthathaventbeencovered
yet,sodontworryifyoudontunderstandallthedetails:
lsl|awk'$6=="Nov"{sum+=$5}
END{printsum}'
35/479
29/7/2016
TheGNUAwkUsersGuide
Thiscommandprintsthetotalnumberofbytesinallthefilesinthecurrentdirectorythatwerelastmodifiedin
November(ofanyyear).Thelslpartofthisexampleisasystemcommandthatgivesyoualistingofthefilesina
directory,includingeachfilessizeandthedatethefilewaslastmodified.Itsoutputlookslikethis:
rwrr1arnolduser1933Nov713:05Makefile
rwrr1arnolduser10809Nov713:03awk.h
rwrr1arnolduser983Apr1312:14awk.tab.h
rwrr1arnolduser31869Jun1512:20awkgram.y
rwrr1arnolduser22414Nov713:03awk1.c
rwrr1arnolduser27511Dec913:07awk3.c
Thefirstfieldcontainsreadwritepermissions,thesecondfieldcontainsthenumberoflinkstothefile,andthethird
fieldidentifiesthefilesowner.Thefourthfieldidentifiesthefilesgroup.Thefifthfieldcontainsthefilessizeinbytes.
Thesixth,seventh,andeighthfieldscontainthemonth,day,andtime,respectively,thatthefilewaslastmodified.
Finally,theninthfieldcontainsthefilename.
The$6=="Nov"inourawkprogramisanexpressionthattestswhetherthesixthfieldoftheoutputfromlsl
matchesthestringNov.EachtimealinehasthestringNovforitssixthfield,awkperformstheactionsum+=$5.
Thisaddsthefifthfield(thefilessize)tothevariablesum.Asaresult,whenawkhasfinishedreadingalltheinputlines,
sumisthetotalofthesizesofthefileswhoselinesmatchedthepattern.(Thisworksbecauseawkvariablesare
automaticallyinitializedtozero.)
Afterthelastlineofoutputfromlshasbeenprocessed,theENDruleexecutesandprintsthevalueofsum.Inthis
example,thevalueofsumis80600.
Thesemoreadvancedawktechniquesarecoveredinlatersections(seeActionOverview).Beforeyoucanmoveonto
moreadvancedawkprogramming,youhavetoknowhowawkinterpretsyourinputanddisplaysyouroutput.By
manipulatingfieldsandusingprintstatements,youcanproducesomeveryusefulandimpressivelookingreports.
Next:OtherFeatures,Previous:MoreComplex,Up:GettingStarted[Contents][Index]
1.6awkStatementsVersusLines
Mostoften,eachlineinanawkprogramisaseparatestatementorseparaterule,likethis:
awk'/12/{print$0}
/21/{print$0}'maillistinventoryshipped
However,gawkignoresnewlinesafteranyofthefollowingsymbolsandkeywords:
,{?:||&&doelse
Anewlineatanyotherpointisconsideredtheendofthestatement.9
Ifyouwouldliketosplitasinglestatementintotwolinesatapointwhereanewlinewouldterminateit,youcan
continueitbyendingthefirstlinewithabackslashcharacter(\).Thebackslashmustbethefinalcharacterontheline
inordertoberecognizedasacontinuationcharacter.Abackslashisallowedanywhereinthestatement,eveninthe
middleofastringorregularexpression.Forexample:
awk'/Thisregularexpressionistoolong,socontinueit\
onthenextline/{print$1}'
Wehavegenerallynotusedbackslashcontinuationinoursampleprograms.gawkplacesnolimitonthelengthofaline,
sobackslashcontinuationisneverstrictlynecessaryitjustmakesprogramsmorereadable.Forthissamereason,as
wellasforclarity,wehavekeptmoststatementsshortintheprogramspresentedthroughouttheWebpage.Backslash
continuationismostusefulwhenyourawkprogramisinaseparatesourcefileinsteadofenteredfromthecommand
line.Youshouldalsonotethatmanyawkimplementationsaremoreparticularaboutwhereyoumayusebackslash
continuation.Forexample,theymaynotallowyoutosplitastringconstantusingbackslashcontinuation.Thus,for
36/479
29/7/2016
TheGNUAwkUsersGuide
maximumportabilityofyourawkprograms,itisbestnottosplityourlinesinthemiddleofaregularexpressionora
string.
CAUTION:BackslashcontinuationdoesnotworkasdescribedwiththeCshell.Itworksforawkprograms
infilesandforoneshotprograms,providedyouareusingaPOSIXcompliantshell,suchastheUnix
BourneshellorBash.ButtheCshellbehavesdifferently!Thereyoumustusetwobackslashesinarow,
followedbyanewline.NotealsothatwhenusingtheCshell,everynewlineinyourawkprogrammustbe
escapedwithabackslash.Toillustrate:
%awk'BEGIN{\
?print\\
?"hello,world"\
?}'
|hello,world
Here,the%and?aretheCshellsprimaryandsecondaryprompts,analogoustothestandardshells$
and>.
ComparethepreviousexampletohowitisdonewithaPOSIXcompliantshell:
$awk'BEGIN{
>print\
>"hello,world"
>}'
|hello,world
awkisalineorientedlanguage.Eachrulesactionhastobeginonthesamelineasthepattern.Tohavethepatternand
actiononseparatelines,youmustusebackslashcontinuationthereisnootheroption.
Anotherthingtokeepinmindisthatbackslashcontinuationandcommentsdonotmix.Assoonasawkseesthe#that
startsacomment,itignoreseverythingontherestoftheline.Forexample:
$gawk'BEGIN{print"dontpanic"#afriendly\
>BEGINrule
>}'
errorgawk:cmd.line:2:BEGINrule
errorgawk:cmd.line:2:^syntaxerror
Inthiscase,itlookslikethebackslashwouldcontinuethecommentontothenextline.However,thebackslashnewline
combinationisneverevennoticedbecauseitishiddeninsidethecomment.Thus,theBEGINisnotedasasyntaxerror.
Whenawkstatementswithinoneruleareshort,youmightwanttoputmorethanoneofthemonaline.Thisis
accomplishedbyseparatingthestatementswithasemicolon(;).Thisalsoappliestotherulesthemselves.Thus,the
programshownatthestartofthissectioncouldalsobewrittenthisway:
/12/{print$0};/21/{print$0}
NOTE:Therequirementthatstatesthatrulesonthesamelinemustbeseparatedwithasemicolonwasnot
intheoriginalawklanguageitwasaddedforconsistencywiththetreatmentofstatementswithinanaction.
Next:When,Previous:Statements/Lines,Up:GettingStarted[Contents][Index]
1.7OtherFeaturesofawk
Theawklanguageprovidesanumberofpredefined,orbuiltin,variablesthatyourprogramscanusetogetinformation
fromawk.Thereareothervariablesyourprogramcansetaswelltocontrolhowawkprocessesyourdata.
Inaddition,awkprovidesanumberofbuiltinfunctionsfordoingcommoncomputationalandstringrelatedoperations.
gawkprovidesbuiltinfunctionsforworkingwithtimestamps,performingbitmanipulation,forruntimestring
translation(internationalization),determiningthetypeofavariable,andarraysorting.
37/479
29/7/2016
TheGNUAwkUsersGuide
Aswedevelopourpresentationoftheawklanguage,wewillintroducemostofthevariablesandmanyofthefunctions.
TheyaredescribedsystematicallyinBuiltinVariables,andinBuiltin.
Next:IntroSummary,Previous:OtherFeatures,Up:GettingStarted[Contents][Index]
1.8WhentoUseawk
Nowthatyouveseensomeofwhatawkcando,youmightwonderhowawkcouldbeusefulforyou.Byusingutility
programs,advancedpatterns,fieldseparators,arithmeticstatements,andotherselectioncriteria,youcanproducemuch
morecomplexoutput.Theawklanguageisveryusefulforproducingreportsfromlargeamountsofrawdata,suchas
summarizinginformationfromtheoutputofotherutilityprogramslikels.(SeeMoreComplex.)
Programswrittenwithawkareusuallymuchsmallerthantheywouldbeinotherlanguages.Thismakesawkprograms
easytocomposeanduse.Often,awkprogramscanbequicklycomposedatyourkeyboard,usedonce,andthrownaway.
Becauseawkprogramsareinterpreted,youcanavoidthe(usuallylengthy)compilationpartofthetypicaleditcompile
testdebugcycleofsoftwaredevelopment.
Complexprogramshavebeenwritteninawk,includingacompleteretargetableassemblerforeightbitmicroprocessors
(seeGlossary,formoreinformation),andamicrocodeassemblerforaspecialpurposePrologcomputer.Theoriginal
awkscapabilitieswerestrainedbytasksofsuchcomplexity,butmodernversionsaremorecapable.
Ifyoufindyourselfwritingawkscriptsofmorethan,say,afewhundredlines,youmightconsiderusingadifferent
programminglanguage.Theshellisgoodatstringandpatternmatchinginaddition,itallowspowerfuluseofthe
systemutilities.Pythonoffersanicebalancebetweenhighleveleaseofprogrammingandaccesstosystemfacilities.10
Previous:When,Up:GettingStarted[Contents][Index]
1.9Summary
Programsinawkconsistofpatternactionpairs.
Anactionwithoutapatternalwaysruns.Thedefaultactionforapatternwithoutoneis{print$0}.
Useeitherawk'program'filesor awkfprogramfilefilestorun awk.
Youmayusethespecial#!headerlinetocreateawkprogramsthataredirectlyexecutable.
Commentsinawkprogramsstartwith#andcontinuetotheendofthesameline.
Beawareofquotingissueswhenwritingawkprogramsaspartofalargershellscript(orMSWindowsbatchfile).
Youmayusebackslashcontinuationtocontinueasourceline.Linesareautomaticallycontinuedafteracomma,
openbrace,questionmark,colon,||,&&,do,andelse.
Next:Regexp,Previous:GettingStarted,Up:Top[Contents][Index]
2Runningawkandgawk
Thischaptercovershowtorunawk,bothPOSIXstandardandgawkspecificcommandlineoptions,andwhatawkand
gawkdowithnonoptionarguments.Itthenproceedstocoverhowgawksearchesforsourcefiles,readingstandardinput
alongwithotherfiles,gawksenvironmentvariables,gawksexitstatus,usingincludefiles,andobsoleteand
undocumentedoptionsand/orfeatures.
ManyoftheoptionsandfeaturesdescribedherearediscussedinmoredetaillaterintheWebpagefeelfreetoskipover
thingsinthischapterthatdontinterestyourightnow.
CommandLine:
Options:
OtherArguments:
Howtorunawk.
Commandlineoptionsandtheirmeanings.
Inputfilenamesandvariableassignments.
38/479
29/7/2016
TheGNUAwkUsersGuide
NamingStandardInput: Howtospecifystandardinputwithotherfiles.
EnvironmentVariables: Theenvironmentvariablesgawkuses.
ExitStatus:
gawksexitstatus.
IncludeFiles:
Includingotherfilesintoyourprogram.
LoadingSharedLibraries: Loadingsharedlibrariesintoyourprogram.
Obsolete:
ObsoleteOptionsand/orfeatures.
Undocumented:
UndocumentedOptionsandFeatures.
InvokingSummary:
Invocationsummary.
Next:Options,Up:InvokingGawk[Contents][Index]
2.1Invokingawk
Therearetwowaystorunawkwithanexplicitprogramorwithoneormoreprogramfiles.Herearetemplatesforboth
ofthemitemsenclosedin[]inthesetemplatesareoptional:
awk[options]fprogfile[]file
awk[options][]'program'file
InadditiontotraditionaloneletterPOSIXstyleoptions,gawkalsosupportsGNUlongoptions.
Itispossibletoinvokeawkwithanemptyprogram:
awk''datafile1datafile2
Doingsomakeslittlesense,thoughawkexitssilentlywhengivenanemptyprogram.(d.c.)Iflinthasbeenspecified
onthecommandline,gawkissuesawarningthattheprogramisempty.
Next:OtherArguments,Previous:CommandLine,Up:InvokingGawk[Contents][Index]
2.2CommandLineOptions
Optionsbeginwithadashandconsistofasinglecharacter.GNUstylelongoptionsconsistoftwodashesanda
keyword.Thekeywordcanbeabbreviated,aslongastheabbreviationallowstheoptiontobeuniquelyidentified.Ifthe
optiontakesanargument,eitherthekeywordisimmediatelyfollowedbyanequalssign(=)andtheargumentsvalue,
orthekeywordandtheargumentsvalueareseparatedbywhitespace.Ifaparticularoptionwithavalueisgivenmore
thanonce,itisthelastvaluethatcounts.
EachlongoptionforgawkhasacorrespondingPOSIXstyleshortoption.Thelongandshortoptionsareinterchangeable
inallcontexts.ThefollowinglistdescribesoptionsmandatedbythePOSIXstandard:
Ffs
fieldseparatorfs
SettheFSvariabletofs(seeFieldSeparators).
fsourcefile
filesourcefile
Readtheawkprogramsourcefromsourcefileinsteadofinthefirstnonoptionargument.Thisoptionmaybe
givenmultipletimestheawkprogramconsistsoftheconcatenationofthecontentsofeachspecifiedsourcefile.
vvar=val
assignvar=val
Setthevariablevartothevaluevalbeforeexecutionoftheprogrambegins.Suchvariablevaluesareavailable
insidetheBEGINrule(seeOtherArguments).
39/479
29/7/2016
TheGNUAwkUsersGuide
Thevoptioncanonlysetonevariable,butitcanbeusedmorethanonce,settinganothervariableeachtime,like
this:awkvfoo=1vbar=2.
CAUTION:Usingvtosetthevaluesofthebuiltinvariablesmayleadtosurprisingresults.awk
willresetthevaluesofthosevariablesasitneedsto,possiblyignoringanyinitialvalueyoumayhave
given.
Wgawkopt
Provideanimplementationspecificoption.ThisisthePOSIXconventionforprovidingimplementationspecific
options.TheseoptionsalsohavecorrespondingGNUstylelongoptions.Notethatthelongoptionsmaybe
abbreviated,aslongastheabbreviationsremainunique.Thefulllistofgawkspecificoptionsisprovidednext.
Signaltheendofthecommandlineoptions.Thefollowingargumentsarenottreatedasoptionsevenifthey
beginwith.ThisinterpretationoffollowsthePOSIXargumentparsingconventions.
Thisisusefulifyouhavefilenamesthatstartwith,orinshellscripts,ifyouhavefilenamesthatwillbe
specifiedbytheuserthatcouldstartwith.Itisalsousefulforpassingoptionsontotheawkprogramsee
GetoptFunction.
Thefollowinglistdescribesgawkspecificoptions:
b
charactersasbytes
Causegawktotreatallinputdataassinglebytecharacters.Inaddition,alloutputwrittenwithprintorprintfis
treatedassinglebytecharacters.
Normally,gawkfollowsthePOSIXstandardandattemptstoprocessitsinputdataaccordingtothecurrentlocale
(seeLocales).Thiscanofteninvolveconvertingmultibytecharactersintowidecharacters(internally),andcan
leadtoproblemsorconfusioniftheinputdatadoesnotcontainvalidmultibytecharacters.Thisoptionisaneasy
waytotellgawk,Handsoffmydata!
c
traditional
Specifycompatibilitymode,inwhichtheGNUextensionstotheawklanguagearedisabled,sothatgawkbehaves
justlikeBWKawk.SeePOSIX/GNU,whichsummarizestheextensions.AlsoseeCompatibilityMode.
C
copyright
PrinttheshortversionoftheGeneralPublicLicenseandthenexit.
d[file]
dumpvariables[=file]
Printasortedlistofglobalvariables,theirtypes,andfinalvaluestofile.Ifnofileisprovided,printthislisttoa
filenamedawkvars.outinthecurrentdirectory.Nospaceisallowedbetweenthedandfile,iffileissupplied.
Havingalistofallglobalvariablesisagoodwaytolookfortypographicalerrorsinyourprograms.Youwould
alsousethisoptionifyouhavealargeprogramwithalotoffunctions,andyouwanttobesurethatyour
functionsdontinadvertentlyuseglobalvariablesthatyoumeanttobelocal.(Thisisaparticularlyeasymistake
tomakewithsimplevariablenameslikei,j,etc.)
D[file]
debug[=file]
Enabledebuggingofawkprograms(seeDebugging).Bydefault,thedebuggerreadscommandsinteractivelyfrom
40/479
29/7/2016
TheGNUAwkUsersGuide
thekeyboard(standardinput).Theoptionalfileargumentallowsyoutospecifyafilewithalistofcommandsfor
thedebuggertoexecutenoninteractively.NospaceisallowedbetweentheDandfile,iffileissupplied.
eprogramtext
sourceprogramtext
Provideprogramsourcecodeintheprogramtext.Thisoptionallowsyoutomixsourcecodeinfileswithsource
codethatyouenteronthecommandline.Thisisparticularlyusefulwhenyouhavelibraryfunctionsthatyou
wanttousefromyourcommandlineprograms(seeAWKPATHVariable).
Efile
execfile
Similartof,readawkprogramtextfromfile.Therearetwodifferencesfromf:
Thisoptionterminatesoptionprocessinganythingelseonthecommandlineispassedondirectlytothe
awkprogram.
Commandlinevariableassignmentsoftheformvar=valuearedisallowed.
ThisoptionisparticularlynecessaryforWorldWideWebCGIapplicationsthatpassargumentsthroughtheURL
usingthisoptionpreventsamalicious(orother)userfrompassinginoptions,assignments,orawksourcecode
(viae)totheCGIapplication.11Thisoptionshouldbeusedwith#!scripts(seeExecutableScripts),likeso:
#!/usr/local/bin/gawkE
awkprogramhere
g
genpot
AnalyzethesourceprogramandgenerateaGNUgettextportableobjecttemplatefileonstandardoutputforall
stringconstantsthathavebeenmarkedfortranslation.SeeInternationalization,forinformationaboutthisoption.
h
help
Printausagemessagesummarizingtheshortandlongstyleoptionsthatgawkacceptsandthenexit.
isourcefile
includesourcefile
Readanawksourcelibraryfromsourcefile.Thisoptioniscompletelyequivalenttousingthe@includedirective
insideyourprogram.Itisverysimilartothefoption,buttherearetwoimportantdifferences.First,wheniis
used,theprogramsourceisnotloadedifithasbeenpreviouslyloaded,whereaswithf,gawkalwaysloadsthe
file.Second,becausethisoptionisintendedtobeusedwithcodelibraries,gawkdoesnotrecognizesuchfilesas
constitutingmainprograminput.Thus,afterprocessinganiargument,gawkstillexpectstofindthemainsource
codeviathefoptionoronthecommandline.
lext
loadext
Loadadynamicextensionnamedext.Extensionsarestoredassystemsharedlibraries.Thisoptionsearchesfor
thelibraryusingtheAWKLIBPATHenvironmentvariable.Thecorrectlibrarysuffixforyourplatformwillbe
suppliedbydefault,soitneednotbespecifiedintheextensionname.Theextensioninitializationroutineshould
benameddl_load().Analternativeistousethe@loadkeywordinsidetheprogramtoloadasharedlibrary.This
advancedfeatureisdescribedindetailinDynamicExtensions.
L[value]
lint[=value]
41/479
29/7/2016
TheGNUAwkUsersGuide
Warnaboutconstructsthataredubiousornonportabletootherawkimplementations.Nospaceisallowedbetween
theLandvalue,ifvalueissupplied.Somewarningsareissuedwhengawkfirstreadsyourprogram.Othersare
issuedatruntime,asyourprogramexecutes.Withanoptionalargumentoffatal,lintwarningsbecomefatal
errors.Thismaybedrastic,butitsusewillcertainlyencouragethedevelopmentofcleanerawkprograms.Withan
optionalargumentofinvalid,onlywarningsaboutthingsthatareactuallyinvalidareissued.(Thisisnotfully
implementedyet.)
Somewarningsareonlyprintedonce,evenifthedubiousconstructstheywarnaboutoccurmultipletimesinyour
awkprogram.Thus,wheneliminatingproblemspointedoutbylint,youshouldtakecaretosearchforall
occurrencesofeachinappropriateconstruct.Asawkprogramsareusuallyshort,doingsoisnotburdensome.
M
bignum
Forcearbitraryprecisionarithmeticonnumbers.ThisoptionhasnoeffectifgawkisnotcompiledtousetheGNU
MPFRandMPlibraries(seeArbitraryPrecisionArithmetic).
n
nondecimaldata
Enableautomaticinterpretationofoctalandhexadecimalvaluesininputdata(seeNondecimalData).
CAUTION:Thisoptioncanseverelybreakoldprograms.Usewithcare.Alsonotethatthisoption
maydisappearinafutureversionofgawk.
N
uselcnumeric
Forcetheuseofthelocalesdecimalpointcharacterwhenparsingnumericinputdata(seeLocales).
o[file]
prettyprint[=file]
Enableprettyprintingofawkprograms.Bydefault,theoutputprogramiscreatedinafilenamedawkprof.out(see
Profiling).Theoptionalfileargumentallowsyoutospecifyadifferentfilenamefortheoutput.Nospaceis
allowedbetweentheoandfile,iffileissupplied.
NOTE:Duetothewaygawkhasevolved,withthisoptionyourprogramstillexecutes.Thiswill
changeinthenextmajorrelease,suchthatgawkwillonlyprettyprinttheprogramandnotrunit.
O
optimize
Enablesomeoptimizationsontheinternalrepresentationoftheprogram.Atthemoment,thisincludesjustsimple
constantfolding.
p[file]
profile[=file]
Enableprofilingofawkprograms(seeProfiling).Bydefault,profilesarecreatedinafilenamedawkprof.out.The
optionalfileargumentallowsyoutospecifyadifferentfilenamefortheprofilefile.Nospaceisallowedbetween
thepandfile,iffileissupplied.
Theprofilecontainsexecutioncountsforeachstatementintheprogramintheleftmargin,andfunctioncall
countsforeachfunction.
P
posix
OperateinstrictPOSIXmode.Thisdisablesallgawkextensions(justliketraditional)anddisablesall
extensionsnotallowedbyPOSIX.SeeCommonExtensions,forasummaryoftheextensionsingawkthatare
42/479
29/7/2016
TheGNUAwkUsersGuide
disabledbythisoption.Also,thefollowingadditionalrestrictionsapply:
NewlinesdonotactaswhitespacetoseparatefieldswhenFSisequaltoasinglespace(seeFields).
Newlinesarenotallowedafter?or:(seeConditionalExp).
SpecifyingFtonthecommandlinedoesnotsetthevalueofFStobeasingleTABcharacter(seeField
Separators).
Thelocalesdecimalpointcharacterisusedforparsinginputdata(seeLocales).
Ifyousupplybothtraditionalandposixonthecommandline,posixtakesprecedence.gawkissuesa
warningifbothoptionsaresupplied.
r
reinterval
Allowintervalexpressions(seeRegexpOperators)inregexps.Thisisnowgawksdefaultbehavior.Nevertheless,
thisoptionremains(bothforbackwardcompatibilityandforuseincombinationwithtraditional).
S
sandbox
Disablethesystem()function,inputredirectionswithgetline,outputredirectionswithprintandprintf,and
dynamicextensions.Thisisparticularlyusefulwhenyouwanttorunawkscriptsfromquestionablesourcesand
needtomakesurethescriptscantaccessyoursystem(otherthanthespecifiedinputdatafile).
t
lintold
WarnaboutconstructsthatarenotavailableintheoriginalversionofawkfromVersion7Unix(seeV7/SVR3.1).
V
version
Printversioninformationforthisparticularcopyofgawk.Thisallowsyoutodetermineifyourcopyofgawkisup
todatewithrespecttowhatevertheFreeSoftwareFoundationiscurrentlydistributing.Itisalsousefulforbug
reports(seeBugs).
Aslongasprogramtexthasbeensupplied,anyotheroptionsareflaggedasinvalidwithawarningmessagebutare
otherwiseignored.
Incompatibilitymode,asaspecialcase,ifthevalueoffssuppliedtotheFoptionist,thenFSissettotheTAB
character("\t").Thisistrueonlyfortraditionalandnotforposix(seeFieldSeparators).
Thefoptionmaybeusedmorethanonceonthecommandline.Ifitis,awkreadsitsprogramsourcefromallofthe
namedfiles,asiftheyhadbeenconcatenatedtogetherintoonebigfile.Thisisusefulforcreatinglibrariesofawk
functions.Thesefunctionscanbewrittenonceandthenretrievedfromastandardplace,insteadofhavingtobe
includedineachindividualprogram.Theioptionissimilarinthisregard.(AsmentionedinDefinitionSyntax,
functionnamesmustbeunique.)
Withstandardawk,libraryfunctionscanstillbeused,eveniftheprogramisenteredatthekeyboard,byspecifyingf
/dev/tty.Aftertypingyourprogram,typeCtrld(theendoffilecharacter)toterminateit.(Youmayalsousefto
readprogramsourcefromthestandardinput,butthenyouwillnotbeabletoalsousethestandardinputasasourceof
data.)
Becauseitisclumsyusingthestandardawkmechanismstomixsourcefileandcommandlineawkprograms,gawk
providestheeoption.Thisdoesnotrequireyoutopreemptthestandardinputforyoursourcecodeitallowsyouto
easilymixcommandlineandlibrarysourcecode(seeAWKPATHVariable).Aswithf,theeandioptionsmayalso
beusedmultipletimesonthecommandline.
Ifnoforeoptionisspecified,thengawkusesthefirstnonoptioncommandlineargumentasthetextoftheprogram
sourcecode.
43/479
29/7/2016
TheGNUAwkUsersGuide
IftheenvironmentvariablePOSIXLY_CORRECTexists,thengawkbehavesinstrictPOSIXmode,exactlyasifyouhad
suppliedposix.ManyGNUprogramslookforthisenvironmentvariabletosuppressextensionsthatconflictwith
POSIX,butgawkbehavesdifferently:itsuppressesallextensions,eventhosethatdonotconflictwithPOSIX,and
behavesinstrictPOSIXmode.IflintissuppliedonthecommandlineandgawkturnsonPOSIXmodebecauseof
POSIXLY_CORRECT,thenitissuesawarningmessageindicatingthatPOSIXmodeisineffect.Youwouldtypicallysetthis
variableinyourshellsstartupfile.ForaBournecompatibleshell(suchasBash),youwouldaddtheselinestothe
.profilefileinyourhomedirectory:
POSIXLY_CORRECT=true
exportPOSIXLY_CORRECT
ForaCshellcompatibleshell,12youwouldaddthislinetothe.loginfileinyourhomedirectory:
setenvPOSIXLY_CORRECTtrue
HavingPOSIXLY_CORRECTsetisnotrecommendedfordailyuse,butitisgoodfortestingtheportabilityofyourprograms
tootherenvironments.
Next:NamingStandardInput,Previous:Options,Up:InvokingGawk[Contents][Index]
2.3OtherCommandLineArguments
Anyadditionalargumentsonthecommandlinearenormallytreatedasinputfilestobeprocessedintheorderspecified.
However,anargumentthathastheformvar=value,assignsthevaluevaluetothevariablevaritdoesnotspecifyafile
atall.(SeeAssignmentOptions.)Inthefollowingexample,count=1isavariableassignment,notafilename:
awkfprogram.awkfile1count=1file2
AllthecommandlineargumentsaremadeavailabletoyourawkprogramintheARGVarray(seeBuiltinVariables).
Commandlineoptionsandtheprogramtext(ifpresent)areomittedfromARGV.Allotherarguments,includingvariable
assignments,areincluded.AseachelementofARGVisprocessed,gawksetsARGINDtotheindexinARGVofthecurrent
element.
ChangingARGCandARGVinyourawkprogramletsyoucontrolhowawkprocessestheinputfilesthisisdescribedinmore
detailinARGCandARGV.
Thedistinctionbetweenfilenameargumentsandvariableassignmentargumentsismadewhenawkisabouttoopenthe
nextinputfile.Atthatpointinexecution,itchecksthefilenametoseewhetheritisreallyavariableassignmentifso,
awksetsthevariableinsteadofreadingafile.
Therefore,thevariablesactuallyreceivethegivenvaluesafterallpreviouslyspecifiedfileshavebeenread.In
particular,thevaluesofvariablesassignedinthisfashionarenotavailableinsideaBEGINrule(seeBEGIN/END),
becausesuchrulesarerunbeforeawkbeginsscanningtheargumentlist.
Thevariablevaluesgivenonthecommandlineareprocessedforescapesequences(seeEscapeSequences).(d.c.)
Insomeveryearlyimplementationsofawk,whenavariableassignmentoccurredbeforeanyfilenames,theassignment
wouldhappenbeforetheBEGINrulewasexecuted.awksbehaviorwasthusinconsistentsomecommandline
assignmentswereavailableinsidetheBEGINrule,whileotherswerenot.Unfortunately,someapplicationscameto
dependuponthisfeature.Whenawkwaschangedtobemoreconsistent,thevoptionwasaddedtoaccommodate
applicationsthatdependedupontheoldbehavior.
ThevariableassignmentfeatureismostusefulforassigningtovariablessuchasRS,OFS,andORS,whichcontrolinput
andoutputformats,beforescanningthedatafiles.Itisalsousefulforcontrollingstateifmultiplepassesareneeded
overadatafile.Forexample:
awk'pass==1{pass1stuff}
pass==2{pass2stuff}'pass=1mydatapass=2mydata
44/479
29/7/2016
TheGNUAwkUsersGuide
Giventhevariableassignmentfeature,theFoptionforsettingthevalueofFSisnotstrictlynecessary.Itremainsfor
historicalcompatibility.
Next:EnvironmentVariables,Previous:OtherArguments,Up:InvokingGawk[Contents][Index]
2.4NamingStandardInput
Often,youmaywishtoreadstandardinputtogetherwithotherfiles.Forexample,youmaywishtoreadonefile,read
standardinputcomingfromapipe,andthenreadanotherfile.
Thewaytonamethestandardinput,withallversionsofawk,istouseasingle,standaloneminussignordash,.For
example:
some_command|awkfmyprog.awkfile1file2
Here,awkfirstreadsfile1,thenitreadstheoutputofsome_command,andfinallyitreadsfile2.
Youmayalsouse""tonamestandardinputwhenreadingfileswithgetline(seeGetline/File).
Inaddition,gawkallowsyoutospecifythespecialfilename/dev/stdin,bothonthecommandlineandwithgetline.
Someotherversionsofawkalsosupportthis,butitisnotstandard.(Someoperatingsystemsprovidea/dev/stdinfilein
thefilesystemhowever,gawkalwaysprocessesthisfilenameitself.)
Next:ExitStatus,Previous:NamingStandardInput,Up:InvokingGawk[Contents][Index]
2.5TheEnvironmentVariablesgawkUses
Anumberofenvironmentvariablesinfluencehowgawkbehaves.
AWKPATHVariable:
Searchingdirectoriesforawkprograms.
AWKLIBPATHVariable:
Searchingdirectoriesforawksharedlibraries.
OtherEnvironmentVariables: Theenvironmentvariables.
Next:AWKLIBPATHVariable,Up:EnvironmentVariables[Contents][Index]
2.5.1TheAWKPATHEnvironmentVariable
Inmostawkimplementations,youmustsupplyaprecisepathnameforeachprogramfile,unlessthefileisinthecurrent
directory.Butwithgawk,ifthefilenamesuppliedtotheforioptionsdoesnotcontainadirectoryseparator/,then
gawksearchesalistofdirectories(calledthesearchpath)onebyone,lookingforafilewiththespecifiedname.
Thesearchpathisastringconsistingofdirectorynamesseparatedbycolons.13gawkgetsitssearchpathfromthe
AWKPATHenvironmentvariable.Ifthatvariabledoesnotexist,orifithasanemptyvalue,gawkusesadefaultpath
(describedshortly).
Thesearchpathfeatureisparticularlyhelpfulforbuildinglibrariesofusefulawkfunctions.Thelibraryfilescanbe
placedinastandarddirectoryinthedefaultpathandthenspecifiedonthecommandlinewithashortfilename.
Otherwise,youwouldhavetotypethefullfilenameforeachfile.
Byusingtheiorfoptions,yourcommandlineawkprogramscanusefacilitiesinawklibraryfiles(seeLibrary
Functions).Pathsearchingisnotdoneifgawkisincompatibilitymode.Thisistrueforbothtraditionalandposix.
SeeOptions.
Ifthesourcecodefileisnotfoundaftertheinitialsearch,thepathissearchedagainafteraddingthesuffix.awktothe
filename.
45/479
29/7/2016
TheGNUAwkUsersGuide
gawkspathsearchmechanismissimilartotheshells.(SeeTheBourneAgainSHellmanual.)Ittreatsanullentryinthe
pathasindicatingthecurrentdirectory.(Anullentryisindicatedbystartingorendingthepathwithacolonorby
placingtwocolonsnexttoeachother[::].)
NOTE:Toincludethecurrentdirectoryinthepath,eitherplace.asanentryinthepathorwriteanull
entryinthepath.
Differentpastversionsofgawkwouldalsolookexplicitlyinthecurrentdirectory,eitherbeforeorafterthe
pathsearch.Asofversion4.1.2,thisnolongerhappensifyouwishtolookinthecurrentdirectory,you
mustinclude.eitherasaseparateentryorasanullentryinthesearchpath.
ThedefaultvalueforAWKPATHis.:/usr/local/share/awk.14Since.isincludedatthebeginning,gawksearchesfirstin
thecurrentdirectoryandthenin/usr/local/share/awk.Inpractice,thismeansthatyouwillrarelyneedtochangethe
valueofAWKPATH.
gawkplacesthevalueofthesearchpaththatitusedintoENVIRON["AWKPATH"].Thisprovidesaccesstotheactualsearch
pathvaluefromwithinanawkprogram.
AlthoughyoucanchangeENVIRON["AWKPATH"]withinyourawkprogram,thishasnoeffectontherunningprograms
behavior.Thismakessense:theAWKPATHenvironmentvariableisusedtofindtheprogramsourcefiles.Onceyour
programisrunning,allthefileshavebeenfound,andgawknolongerneedstouseAWKPATH.
Next:OtherEnvironmentVariables,Previous:AWKPATHVariable,Up:EnvironmentVariables[Contents][Index]
2.5.2TheAWKLIBPATHEnvironmentVariable
TheAWKLIBPATHenvironmentvariableissimilartotheAWKPATHvariable,butitisusedtosearchforloadableextensions
(storedassystemsharedlibraries)specifiedwiththeloptionratherthanforsourcefiles.Iftheextensionisnotfound,
thepathissearchedagainafteraddingtheappropriatesharedlibrarysuffixfortheplatform.Forexample,on
GNU/Linuxsystems,thesuffix.soisused.Thesearchpathspecifiedisalsousedforextensionsloadedviathe@load
keyword(seeLoadingSharedLibraries).
IfAWKLIBPATHdoesnotexistintheenvironment,orifithasanemptyvalue,gawkusesadefaultpaththisistypically
/usr/local/lib/gawk,althoughitcanvarydependinguponhowgawkwasbuilt.
gawkplacesthevalueofthesearchpaththatitusedintoENVIRON["AWKLIBPATH"].Thisprovidesaccesstotheactual
searchpathvaluefromwithinanawkprogram.
Previous:AWKLIBPATHVariable,Up:EnvironmentVariables[Contents][Index]
2.5.3OtherEnvironmentVariables
Anumberofotherenvironmentvariablesaffectgawksbehavior,buttheyaremorespecialized.Thoseinthefollowing
listaremeanttobeusedbyregularusers:
GAWK_MSEC_SLEEP
Specifiestheintervalbetweenconnectionretries,inmilliseconds.Onsystemsthatdonotsupporttheusleep()
systemcall,thevalueisroundeduptoanintegralnumberofseconds.
GAWK_READ_TIMEOUT
Specifiesthetime,inmilliseconds,forgawktowaitforinputbeforereturningwithanerror.SeeReadTimeout.
GAWK_SOCK_RETRIES
ControlsthenumberoftimesgawkattemptstoretryatwowayTCP/IP(socket)connectionbeforegivingup.See
46/479
29/7/2016
TheGNUAwkUsersGuide
TCP/IPNetworking.
POSIXLY_CORRECT
CausesgawktoswitchtoPOSIXcompatibilitymode,disablingalltraditionalandGNUextensions.SeeOptions.
Theenvironmentvariablesinthefollowinglistaremeantforusebythegawkdevelopersfortestingandtuning.Theyare
subjecttochange.Thevariablesare:
AWKBUFSIZE
ThisvariableonlyaffectsgawkonPOSIXcompliantsystems.Withavalueofexact,gawkusesthesizeofeach
inputfileasthesizeofthememorybuffertoallocateforI/O.Otherwise,thevalueshouldbeanumber,andgawk
usesthatnumberasthesizeofthebuffertoallocate.(Whenthisvariableisnotset,gawkusesthesmallerofthe
filessizeandthedefaultblocksize,whichisusuallythefilesystemsI/Oblocksize.)
AWK_HASH
Ifthisvariableexistswithavalueofgst,gawkswitchestousingthehashfunctionfromGNUSmalltalkfor
managingarrays.Thisfunctionmaybemarginallyfasterthanthestandardfunction.
AWKREADFUNC
Ifthisvariableexists,gawkswitchestoreadingsourcefilesonelineatatime,insteadofreadinginblocks.This
existsfordebuggingproblemsonfilesystemsonnonPOSIXoperatingsystemswhereI/Oisperformedin
records,notinblocks.
GAWK_MSG_SRC
Ifthisvariableexists,gawkincludesthefilenameandlinenumberwithinthegawksourcecodefromwhich
warningand/orfatalmessagesaregenerated.Itspurposeistohelpisolatethesourceofamessage,asthereare
multipleplacesthatproducethesamewarningorerrormessage.
GAWK_NO_DFA
Ifthisvariableexists,gawkdoesnotusetheDFAregexpmatcherfordoesitmatchkindsoftests.Thiscancause
gawktobeslower.Itspurposeistohelpisolatedifferencesbetweenthetworegexpmatchersthatgawkuses
internally.(Therearentsupposedtobedifferences,butoccasionallytheoryandpracticedontcoordinatewith
eachother.)
GAWK_NO_PP_RUN
Whengawkisinvokedwiththeprettyprintoption,itwillnotruntheprogramifthisenvironmentvariable
exists.
CAUTION:Thisvariablewillnotsurviveintothenextmajorrelease.
GAWK_STACKSIZE
Thisspecifiestheamountbywhichgawkshouldgrowitsinternalevaluationstack,whenneeded.
INT_CHAIN_MAX
Thisspecifiesintendedmaximumnumberofitemsgawkwillmaintainonahashchainformanagingarrays
indexedbyintegers.
STR_CHAIN_MAX
Thisspecifiesintendedmaximumnumberofitemsgawkwillmaintainonahashchainformanagingarrays
indexedbystrings.
TIDYMEM
47/479
29/7/2016
TheGNUAwkUsersGuide
Ifthisvariableexists,gawkusesthemtrace()librarycallsfromtheGNUClibrarytohelptrackdownpossible
memoryleaks.
Next:IncludeFiles,Previous:EnvironmentVariables,Up:InvokingGawk[Contents][Index]
2.6gawksExitStatus
Iftheexitstatementisusedwithavalue(seeExitStatement),thengawkexitswiththenumericvaluegiventoit.
Otherwise,iftherewerenoproblemsduringexecution,gawkexitswiththevalueoftheCconstantEXIT_SUCCESS.Thisis
usuallyzero.
Ifanerroroccurs,gawkexitswiththevalueoftheCconstantEXIT_FAILURE.Thisisusuallyone.
Ifgawkexitsbecauseofafatalerror,theexitstatusistwo.OnnonPOSIXsystems,thisvaluemaybemappedto
EXIT_FAILURE.
Next:LoadingSharedLibraries,Previous:ExitStatus,Up:InvokingGawk[Contents][Index]
2.7IncludingOtherFilesintoYourProgram
Thissectiondescribesafeaturethatisspecifictogawk.
The@includekeywordcanbeusedtoreadexternalawksourcefiles.Thisgivesyoutheabilitytosplitlargeawksource
filesintosmaller,moremanageablepieces,andalsoletsyoureusecommonawkcodefromvariousawkscripts.Inother
words,youcangrouptogetherawkfunctionsusedtocarryoutspecifictasksintoexternalfiles.Thesefilescanbeused
justlikefunctionlibraries,usingthe@includekeywordinconjunctionwiththeAWKPATHenvironmentvariable.Notethat
sourcefilesmayalsobeincludedusingtheioption.
Letsseeanexample.Wellstartwithtwo(trivial)awkscripts,namelytest1andtest2.Hereisthetest1script:
BEGIN{
print"Thisisscripttest1."
}
andhereistest2:
@include"test1"
BEGIN{
}
Runninggawkwithtest2producesthefollowingresult:
$gawkftest2
|Thisisscripttest1.
|Thisisscripttest2.
gawkrunsthetest2script,whichincludestest1usingthe@includekeyword.So,toincludeexternalawksourcefiles,
youjustuse@includefollowedbythenameofthefiletobeincluded,enclosedindoublequotes.
NOTE:Keepinmindthatthisisalanguageconstructandthefilenamecannotbeastringvariable,but
ratherjustaliteralstringconstantindoublequotes.
Thefilestobeincludedmaybenestede.g.,givenathirdscript,namelytest3:
@include"test2"
BEGIN{
48/479
29/7/2016
TheGNUAwkUsersGuide
Runninggawkwiththetest3scriptproducesthefollowingresults:
$gawkftest3
|Thisisscripttest1.
|Thisisscripttest2.
|Thisisscripttest3.
Thefilenamecan,ofcourse,beapathname.Forexample:
@include"../io_funcs"
and:
@include"/usr/awklib/network"
arebothvalid.TheAWKPATHenvironmentvariablecanbeofgreatvaluewhenusing@include.Thesamerulesfortheuse
oftheAWKPATHvariableincommandlinefilesearches(seeAWKPATHVariable)applyto@includealso.
Thisisveryhelpfulinconstructinggawkfunctionlibraries.Ifyouhavealargescriptwithuseful,generalpurposeawk
functions,youcanbreakitdownintolibraryfilesandputthosefilesinaspecialdirectory.Youcanthenincludethose
libraries,eitherbyusingthefullpathnamesofthefiles,orbysettingtheAWKPATHenvironmentvariableaccordingly
andthenusing@includewithjustthefilepartofthefullpathname.Ofcourse,youcankeeplibraryfilesinmorethan
onedirectorythemorecomplextheworkingenvironmentis,themoredirectoriesyoumayneedtoorganizethefilesto
beincluded.
Giventheabilitytospecifymultiplefoptions,the@includemechanismisnotstrictlynecessary.However,the
@includekeywordcanhelpyouinconstructingselfcontainedgawkprograms,thusreducingtheneedforwriting
complexandtediouscommandlines.Inparticular,@includeisveryusefulforwritingCGIscriptstoberunfromweb
pages.
AsmentionedinAWKPATHVariable,thecurrentdirectoryisalwayssearchedfirstforsourcefiles,beforesearchingin
AWKPATHthisalsoappliestofilesnamedwith@include.
Next:Obsolete,Previous:IncludeFiles,Up:InvokingGawk[Contents][Index]
2.8LoadingDynamicExtensionsintoYourProgram
The@loadkeywordcanbeusedtoreadexternalawkextensions(storedassystemsharedlibraries).Thisallowsyouto
linkincompiledcodethatmayoffersuperiorperformanceand/orgiveyouaccesstoextendedcapabilitiesnotsupported
bytheawklanguage.TheAWKLIBPATHvariableisusedtosearchfortheextension.Using@loadiscompletelyequivalent
tousingthelcommandlineoption.
IftheextensionisnotinitiallyfoundinAWKLIBPATH,anothersearchisconductedafterappendingtheplatformsdefault
sharedlibrarysuffixtothefilename.Forexample,onGNU/Linuxsystems,thesuffix.soisused:
$gawk'@load"ordchr";BEGIN{printchr(65)}'
|A
Thisisequivalenttothefollowingexample:
$gawklordchr'BEGIN{printchr(65)}'
|A
Forcommandlineusage,theloptionismoreconvenient,but@loadisusefulforembeddinginsideanawksourcefile
thatrequiresaccesstoanextension.
49/479
29/7/2016
TheGNUAwkUsersGuide
DynamicExtensions,describeshowtowriteextensions(inCorC++)thatcanbeloadedwitheither@loadorthel
option.Italsodescribestheordchrextension.
Next:Undocumented,Previous:LoadingSharedLibraries,Up:InvokingGawk[Contents][Index]
2.9ObsoleteOptionsand/orFeatures
Thissectiondescribesfeaturesand/orcommandlineoptionsfrompreviousreleasesofgawkthateitherarenotavailable
inthecurrentversionorarestillsupportedbutdeprecated(meaningthattheywillnotbeinthenextrelease).
Theprocessrelatedspecialfiles/dev/pid,/dev/ppid,/dev/pgrpid,and/dev/userweredeprecatedingawk3.1,butstill
worked.Asofversion4.0,theyarenolongerinterpretedspeciallybygawk.(UsePROCINFOinsteadseeAutoset.)
Next:InvokingSummary,Previous:Obsolete,Up:InvokingGawk[Contents][Index]
2.10UndocumentedOptionsandFeatures
UsetheSource,Luke!
ObiWan
Thissectionintentionallyleftblank.
Previous:Undocumented,Up:InvokingGawk[Contents][Index]
2.11Summary
Useeitherawk'program'filesor awkfprogramfilefilestorun awk.
Thethreestandardoptionsforallversionsofawkaref,F,andv.gawksuppliestheseandmanyothers,aswell
ascorrespondingGNUstylelongoptions.
Nonoptioncommandlineargumentsareusuallytreatedasfilenames,unlesstheyhavetheformvar=value,in
whichcasetheyaretakenasvariableassignmentstobeperformedatthatpointinprocessingtheinput.
Allnonoptioncommandlinearguments,excludingtheprogramtext,areplacedintheARGVarray.AdjustingARGC
andARGVaffectshowawkprocessesinput.
Youcanuseasingleminussign()torefertostandardinputonthecommandline.gawkalsoletsyouusethe
specialfilename/dev/stdin.
gawkpaysattentiontoanumberofenvironmentvariables.AWKPATH,AWKLIBPATH,andPOSIXLY_CORRECTarethemost
importantones.
gawksexitstatusconveysinformationtotheprogramthatinvokedit.Usetheexitstatementfromwithinanawk
programtosettheexitstatus.
gawkallowsyoutoincludeotherawksourcefilesintoyourprogramusingthe@includestatementand/orthei
andfcommandlineoptions.
gawkallowsyoutoloadadditionalfunctionswritteninCorC++usingthe@loadstatementand/ortheloption.
(Thisadvancedfeatureisdescribedlater,inDynamicExtensions.)
Next:ReadingFiles,Previous:InvokingGawk,Up:Top[Contents][Index]
3RegularExpressions
Aregularexpression,orregexp,isawayofdescribingasetofstrings.Becauseregularexpressionsaresucha
fundamentalpartofawkprogramming,theirformatandusedeserveaseparatechapter.
Aregularexpressionenclosedinslashes(/)isanawkpatternthatmatcheseveryinputrecordwhosetextbelongsto
50/479
29/7/2016
TheGNUAwkUsersGuide
thatset.Thesimplestregularexpressionisasequenceofletters,numbers,orboth.Sucharegexpmatchesanystring
thatcontainsthatsequence.Thus,theregexpfoomatchesanystringcontainingfoo.Thus,thepattern/foo/matches
anyinputrecordcontainingthethreeadjacentcharactersfooanywhereintherecord.Otherkindsofregexpsletyou
specifymorecomplicatedclassesofstrings.
Initially,theexamplesinthischapteraresimple.Asweexplainmoreabouthowregularexpressionswork,wepresent
morecomplicatedinstances.
RegexpUsage:
HowtoUseRegularExpressions.
EscapeSequences:
Howtowritenonprintingcharacters.
RegexpOperators:
RegularExpressionOperators.
BracketExpressions:
Whatcangobetween[...].
LeftmostLongest:
Howmuchtextmatches.
ComputedRegexps:
UsingDynamicRegexps.
GNURegexpOperators: OperatorsspecifictoGNUsoftware.
Casesensitivity:
Howtodocaseinsensitivematching.
RegexpSummary:
Regularexpressionssummary.
Next:EscapeSequences,Up:Regexp[Contents][Index]
3.1HowtoUseRegularExpressions
Aregularexpressioncanbeusedasapatternbyenclosingitinslashes.Thentheregularexpressionistestedagainstthe
entiretextofeachrecord.(Normally,itonlyneedstomatchsomepartofthetextinordertosucceed.)Forexample,the
followingprintsthesecondfieldofeachrecordwherethestringliappearsanywhereintherecord:
$awk'/li/{print$2}'maillist
|5555553
|5550542
|5556699
|5553430
Regularexpressionscanalsobeusedinmatchingexpressions.Theseexpressionsallowyoutospecifythestringto
matchagainstitneednotbetheentirecurrentinputrecord.Thetwooperators~and !~performregularexpression
comparisons.Expressionsusingtheseoperatorscanbeusedaspatterns,orinif,while,for,anddostatements.(See
Statements.)Forexample,thefollowingistrueiftheexpressionexp(takenasastring)matchesregexp:
exp~/regexp/
Thisexamplematches,orselects,allinputrecordswiththeuppercaseletterJsomewhereinthefirstfield:
$awk'$1~/J/'inventoryshipped
|Jan132515115
|Jun314275492
|Jul243467436
|Jan213664620
Sodoesthis:
awk'{if($1~/J/)print}'inventoryshipped
Thisnextexampleistrueiftheexpressionexp(takenasacharacterstring)doesnotmatchregexp:
exp!~/regexp/
Thefollowingexamplematches,orselects,allinputrecordswhosefirstfielddoesnotcontaintheuppercaseletterJ:
$awk'$1!~/J/'inventoryshipped
|Feb153224226
|Mar152434228
51/479
29/7/2016
TheGNUAwkUsersGuide
|Apr315263420
|May163429208
Whenaregexpisenclosedinslashes,suchas/foo/,wecallitaregexpconstant,muchlike5.27isanumericconstant
and"foo"isastringconstant.
Next:RegexpOperators,Previous:RegexpUsage,Up:Regexp[Contents][Index]
3.2EscapeSequences
Somecharacterscannotbeincludedliterallyinstringconstants("foo")orregexpconstants(/foo/).Instead,theyshould
berepresentedwithescapesequences,whicharecharactersequencesbeginningwithabackslash(\).Oneuseofan
escapesequenceistoincludeadoublequotecharacterinastringconstant.Becauseaplaindoublequoteendsthestring,
youmustuse\"torepresentanactualdoublequotecharacterasapartofthestring.Forexample:
$awk'BEGIN{print"Hesaid\"hi!\"toher."}'
|Hesaid"hi!"toher.
Thebackslashcharacteritselfisanothercharacterthatcannotbeincludednormallyyoumustwrite\\toputone
backslashinthestringorregexp.Thus,thestringwhosecontentsarethetwocharacters"and\mustbewritten
"\"\\".
OtherescapesequencesrepresentunprintablecharacterssuchasTABornewline.Thereisnothingtostopyoufrom
enteringmostunprintablecharactersdirectlyinastringconstantorregexpconstant,buttheymaylookugly.
Thefollowinglistpresentsalltheescapesequencesusedinawkandwhattheyrepresent.Unlessnotedotherwise,all
theseescapesequencesapplytobothstringconstantsandregexpconstants:
\\
Aliteralbackslash,\.
\a
Thealertcharacter,Ctrlg,ASCIIcode7(BEL).(Thisoftenmakessomesortofaudiblenoise.)
\b
Backspace,Ctrlh,ASCIIcode8(BS).
\f
Formfeed,Ctrll,ASCIIcode12(FF).
\n
Newline,Ctrlj,ASCIIcode10(LF).
\r
Carriagereturn,Ctrlm,ASCIIcode13(CR).
\t
HorizontalTAB,Ctrli,ASCIIcode9(HT).
\v
VerticalTAB,Ctrlk,ASCIIcode11(VT).
52/479
29/7/2016
TheGNUAwkUsersGuide
\nnn
Theoctalvaluennn,wherennnstandsfor1to3digitsbetween0and7.Forexample,thecodefortheASCII
ESC(escape)characteris\033.
\xhh
Thehexadecimalvaluehh,wherehhstandsforasequenceofhexadecimaldigits(09,andeitherAFor
af).LikethesameconstructinISOC,theescapesequencecontinuesuntilthefirstnonhexadecimaldigitis
seen.(c.e.)However,usingmorethantwohexadecimaldigitsproducesundefinedresults.(The\xescape
sequenceisnotallowedinPOSIXawk.)
CAUTION:Thenextmajorreleaseofgawkwillchange,suchthatamaximumoftwohexadecimal
digitsfollowingthe\xwillbeused.
\/
Aliteralslash(necessaryforregexpconstantsonly).Thissequenceisusedwhenyouwanttowritearegexp
constantthatcontainsaslash(suchas/.*:\/home\/[[:alnum:]]+:.*/the[[:alnum:]]notationisdiscussedin
BracketExpressions).Becausetheregexpisdelimitedbyslashes,youneedtoescapeanyslashthatispartofthe
pattern,inordertotellawktokeepprocessingtherestoftheregexp.
\"
Aliteraldoublequote(necessaryforstringconstantsonly).Thissequenceisusedwhenyouwanttowriteastring
constantthatcontainsadoublequote(suchas"Hesaid\"hi!\"toher.").Becausethestringisdelimitedby
doublequotes,youneedtoescapeanyquotethatispartofthestring,inordertotellawktokeepprocessingthe
restofthestring.
Ingawk,anumberofadditionaltwocharactersequencesthatbeginwithabackslashhavespecialmeaninginregexps.
SeeGNURegexpOperators.
Inaregexp,abackslashbeforeanycharacterthatisnotinthepreviouslistandnotlistedinGNURegexpOperators,
meansthatthenextcharactershouldbetakenliterally,evenifitwouldnormallybearegexpoperator.Forexample,
/a\+b/matchesthethreecharactersa+b.
Forcompleteportability,donotuseabackslashbeforeanycharacternotshowninthepreviouslistorthatisnotan
operator.
BackslashBeforeRegularCharacters
Ifyouplaceabackslashinastringconstantbeforesomethingthatisnotoneofthecharacterspreviouslylisted,
POSIXawkpurposelyleaveswhathappensasundefined.Therearetwochoices:
Stripthebackslashout
ThisiswhatBWKawkandgawkbothdo.Forexample,"a\qc"isthesameas"aqc".(Becausethisissuchaneasy
bugbothtointroduceandtomiss,gawkwarnsyouaboutit.)ConsiderFS="[\t]+\|[\t]+"tousevertical
barssurroundedbywhitespaceasthefieldseparator.Thereshouldbetwobackslashesinthestring:FS="
[\t]+\\|[\t]+".)
Leavethebackslashalone
Someotherawkimplementationsdothis.Insuchimplementations,typing"a\qc"isthesameastyping"a\\qc".
Tosummarize:
Theescapesequencesintheprecedinglistarealwaysprocessedfirst,forbothstringconstantsandregexp
constants.Thishappensveryearly,assoonasawkreadsyourprogram.
53/479
29/7/2016
TheGNUAwkUsersGuide
gawkprocessesbothregexpconstantsanddynamicregexps(seeComputedRegexps),forthespecialoperators
listedinGNURegexpOperators.
Abackslashbeforeanyothercharactermeanstotreatthatcharacterliterally.
EscapeSequencesforMetacharacters
Supposeyouuseanoctalorhexadecimalescapetorepresentaregexpmetacharacter.(SeeRegexpOperators.)Does
awktreatthecharacterasaliteralcharacterorasaregexpoperator?
Historically,suchcharactersweretakenliterally.(d.c.)However,thePOSIXstandardindicatesthattheyshouldbe
treatedasrealmetacharacters,whichiswhatgawkdoes.Incompatibilitymode(seeOptions),gawktreatsthecharacters
representedbyoctalandhexadecimalescapesequencesliterallywhenusedinregexpconstants.Thus,/a\52b/is
equivalentto/a\*b/.
Next:BracketExpressions,Previous:EscapeSequences,Up:Regexp[Contents][Index]
3.3RegularExpressionOperators
Youcancombineregularexpressionswithspecialcharacters,calledregularexpressionoperatorsormetacharacters,to
increasethepowerandversatilityofregularexpressions.
TheescapesequencesdescribedearlierinEscapeSequences,arevalidinsidearegexp.Theyareintroducedbya\and
arerecognizedandconvertedintocorrespondingrealcharactersastheveryfirststepinprocessingregexps.
Hereisalistofmetacharacters.Allcharactersthatarenotescapesequencesandthatarenotlistedherestandfor
themselves:
\
Thissuppressesthespecialmeaningofacharacterwhenmatching.Forexample,\$matchesthecharacter $.
^
Thismatchesthebeginningofastring.^@chaptermatches @chapteratthebeginningofastring,forexample,
andcanbeusedtoidentifychapterbeginningsinTexinfosourcefiles.Theîsknownasananchor,becauseit
anchorsthepatterntomatchonlyatthebeginningofthestring.
Itisimportanttorealizethat^doesnotmatchthebeginningofaline(thepointrightaftera\nnewline
character)embeddedinastring.Theconditionisnottrueinthefollowingexample:
if("line1\nLINE2"~/^L/)
$
Thisissimilarto^,butitmatchesonlyattheendofastring.Forexample,p$matchesarecordthatendswith
ap.The$isananchoranddoesnotmatchtheendofaline(thepointrightbeforea\nnewlinecharacter)
embeddedinastring.Theconditioninthefollowingexampleisnottrue:
if("line1\nLINE2"~/1$/)
.(period)
Thismatchesanysinglecharacter,includingthenewlinecharacter.Forexample,.Pmatchesanysingle
characterfollowedbyaPinastring.Usingconcatenation,wecanmakearegularexpressionsuchasU.A,
whichmatchesanythreecharactersequencethatbeginswithUandendswithA.
InstrictPOSIXmode(seeOptions),.doesnotmatchtheNULcharacter,whichisacharacterwithallbitsequal
tozero.Otherwise,NULisjustanothercharacter.OtherversionsofawkmaynotbeabletomatchtheNUL
54/479
29/7/2016
TheGNUAwkUsersGuide
character.
[]
Thisiscalledabracketexpression.15Itmatchesanyoneofthecharactersthatareenclosedinthesquarebrackets.
Forexample,[MVX]matchesanyoneofthecharactersM,V,orXinastring.Afulldiscussionofwhatcanbe
insidethesquarebracketsofabracketexpressionisgiveninBracketExpressions.
[^]
Thisisacomplementedbracketexpression.Thefirstcharacterafterthe[mustbea^.Itmatchesany
charactersexceptthoseinthesquarebrackets.Forexample,[âwk]matchesanycharacterthatisnotan a,w,
ork.
|
Thisisthealternationoperatoranditisusedtospecifyalternatives.The|hasthelowestprecedenceofallthe
regularexpressionoperators.Forexample,^P|[aeiouy]matchesanystringthatmatcheseither^Por
[aeiouy].ThismeansitmatchesanystringthatstartswithPorcontains(anywherewithinit)alowercase
Englishvowel.
Thealternationappliestothelargestpossibleregexpsoneitherside.
()
Parenthesesareusedforgroupinginregularexpressions,asinarithmetic.Theycanbeusedtoconcatenate
regularexpressionscontainingthealternationoperator,|.Forexample,@(samp|code)\{[^}]+\}matchesboth
@code{foo}and@samp{bar}.(TheseareTexinfoformattingcontrolsequences.The+isexplainedfurtheron
inthislist.)
*
Thissymbolmeansthattheprecedingregularexpressionshouldberepeatedasmanytimesasnecessarytofinda
match.Forexample,ph*appliesthe *symboltotheprecedinghandlooksformatchesofonepfollowed
byanynumberofhs.Thisalsomatchesjustpifnohsarepresent.
Therearetwosubtlepointstounderstandabouthow*works.First,the*appliesonlytothesinglepreceding
regularexpressioncomponent(e.g.,inph*,itappliesjusttotheh).Tocause*toapplytoalarger
subexpression,useparentheses:(ph)*matchesph,phph,phphph,andsoon.
Second,*findsasmanyrepetitionsaspossible.Ifthetexttobematchedisphhhhhhhhhhhhhhooey,ph*
matchesallofthehs.
+
Thissymbolissimilarto*,exceptthattheprecedingexpressionmustbematchedatleastonce.Thismeansthat
wh+ywouldmatch whyandwhhy,butnotwy,whereaswh*ywouldmatchallthree.
?
Thissymbolissimilarto*,exceptthattheprecedingexpressioncanbematchedeitheronceornotatall.For
example,fe?dmatches fedandfd,butnothingelse.
{n}
{n,}
{n,m}
Oneortwonumbersinsidebracesdenoteanintervalexpression.Ifthereisonenumberinthebraces,the
precedingregexpisrepeatedntimes.Iftherearetwonumbersseparatedbyacomma,theprecedingregexpis
repeatedntomtimes.Ifthereisonenumberfollowedbyacomma,thentheprecedingregexpisrepeatedatleast
55/479
29/7/2016
TheGNUAwkUsersGuide
ntimes:
wh{3}y
Matcheswhhhy,butnotwhyorwhhhhy.
wh{3,5}y
Matcheswhhhy,whhhhy,orwhhhhhyonly.
wh{2,}y
Matcheswhhy,whhhy,andsoon.
Intervalexpressionswerenottraditionallyavailableinawk.TheywereaddedaspartofthePOSIXstandardto
makeawkandegrepconsistentwitheachother.
Initially,becauseoldprogramsmayuse{and}inregexpconstants,gawkdidnotmatchintervalexpressionsin
regexps.
However,beginningwithversion4.0,gawkdoesmatchintervalexpressionsbydefault.Thisisbecause
compatibilitywithPOSIXhasbecomemoreimportanttomostgawkusersthancompatibilitywitholdprograms.
Forprogramsthatuse{and}inregexpconstants,itisgoodpracticetoalwaysescapethemwithabackslash.
Thentheregexpconstantsarevalidandworkthewayyouwantthemto,usinganyversionofawk.16
Finally,when{and}appearinregexpconstantsinawaythatcannotbeinterpretedasanintervalexpression
(suchas/q{a}/),thentheystandforthemselves.
Inregularexpressions,the*,+,and?operators,aswellasthebraces{and},havethehighestprecedence,
followedbyconcatenation,andfinallyby|.Asinarithmetic,parenthesescanchangehowoperatorsaregrouped.
InPOSIXawkandgawk,the*,+,and?operatorsstandforthemselveswhenthereisnothingintheregexpthat
precedesthem.Forexample,/+/matchesaliteralplussign.However,manyotherversionsofawktreatsuchausageas
asyntaxerror.
Ifgawkisincompatibilitymode(seeOptions),intervalexpressionsarenotavailableinregularexpressions.
Next:LeftmostLongest,Previous:RegexpOperators,Up:Regexp[Contents][Index]
3.4UsingBracketExpressions
Asmentionedearlier,abracketexpressionmatchesanycharacteramongthoselistedbetweentheopeningandclosing
squarebrackets.
Withinabracketexpression,arangeexpressionconsistsoftwocharactersseparatedbyahyphen.Itmatchesanysingle
characterthatsortsbetweenthetwocharacters,baseduponthesystemsnativecharacterset.Forexample,[09]is
equivalentto[0123456789].(SeeRangesandLocales,foranexplanationofhowthePOSIXstandardandgawkhave
changedovertime.Thisismainlyofhistoricalinterest.)
Toincludeoneofthecharacters\,],,orînabracketexpression,puta\infrontofit.Forexample:
[d\]]
matcheseitherdor].Additionally,ifyouplace]rightaftertheopening[,theclosingbracketistreatedasoneof
thecharacterstobematched.
Thetreatmentof\inbracketexpressionsiscompatiblewithotherawkimplementationsandisalsomandatedby
POSIX.TheregularexpressionsinawkareasupersetofthePOSIXspecificationforExtendedRegularExpressions
56/479
29/7/2016
TheGNUAwkUsersGuide
(EREs).POSIXEREsarebasedontheregularexpressionsacceptedbythetraditionalegreputility.
CharacterclassesareafeatureintroducedinthePOSIXstandard.Acharacterclassisaspecialnotationfordescribing
listsofcharactersthathaveaspecificattribute,buttheactualcharacterscanvaryfromcountrytocountryand/orfrom
charactersettocharacterset.Forexample,thenotionofwhatisanalphabeticcharacterdiffersbetweentheUnited
StatesandFrance.
Acharacterclassisonlyvalidinaregexpinsidethebracketsofabracketexpression.Characterclassesconsistof[:,a
keyworddenotingtheclass,and:].Table3.1liststhecharacterclassesdefinedbythePOSIXstandard.
Class
[:alnum:]
[:alpha:]
[:blank:]
[:cntrl:]
[:digit:]
[:graph:]
[:lower:]
[:print:]
[:punct:]
[:space:]
[:upper:]
[:xdigit:]
Meaning
Alphanumericcharacters
Alphabeticcharacters
SpaceandTABcharacters
Controlcharacters
Numericcharacters
Charactersthatarebothprintableandvisible(aspaceisprintablebutnotvisible,whereasanais
both)
Lowercasealphabeticcharacters
Printablecharacters(charactersthatarenotcontrolcharacters)
Punctuationcharacters(charactersthatarenotletters,digits,controlcharacters,orspacecharacters)
Spacecharacters(suchasspace,TAB,andformfeed,tonameafew)
Uppercasealphabeticcharacters
Charactersthatarehexadecimaldigits
Table3.1:POSIXcharacterclasses
Forexample,beforethePOSIXstandard,youhadtowrite/[AZaz09]/tomatchalphanumericcharacters.Ifyour
charactersethadotheralphabeticcharactersinit,thiswouldnotmatchthem.WiththePOSIXcharacterclasses,you
canwrite/[[:alnum:]]/tomatchthealphabeticandnumericcharactersinyourcharacterset.
Someutilitiesthatmatchregularexpressionsprovideanonstandard[:ascii:]characterclassawkdoesnot.However,
youcansimulatesuchaconstructusing[\x00\x7F].Thismatchesallvaluesnumericallybetweenzeroand127,
whichisthedefinedrangeoftheASCIIcharacterset.Useacomplementedcharacterlist([^\x00\x7F])tomatchany
singlebytecharactersthatarenotintheASCIIrange.
Twoadditionalspecialsequencescanappearinbracketexpressions.TheseapplytononASCIIcharactersets,which
canhavesinglesymbols(calledcollatingelements)thatarerepresentedwithmorethanonecharacter.Theycanalso
haveseveralcharactersthatareequivalentforcollating,orsorting,purposes.(Forexample,inFrench,aplaineanda
graveaccentedareequivalent.)Thesesequencesare:
Collatingsymbols
Multicharactercollatingelementsenclosedbetween[.and.].Forexample,ifchisacollatingelement,then
[[.ch.]]isaregexpthatmatchesthiscollatingelement,whereas[ch]isaregexpthatmatcheseithercor
h.
Equivalenceclasses
Localespecificnamesforalistofcharactersthatareequal.Thenameisenclosedbetween[=and=].For
example,thenameemightbeusedtorepresentallofe,,,and.Inthiscase,[[=e=]]isaregexp
thatmatchesanyofe,,,or.
ThesefeaturesareveryvaluableinnonEnglishspeakinglocales.
CAUTION:Thelibraryfunctionsthatgawkusesforregularexpressionmatchingcurrentlyrecognizeonly
57/479
29/7/2016
TheGNUAwkUsersGuide
POSIXcharacterclassestheydonotrecognizecollatingsymbolsorequivalenceclasses.
Next:ComputedRegexps,Previous:BracketExpressions,Up:Regexp[Contents][Index]
3.5HowMuchTextMatches?
Considerthefollowing:
echoaaaabcd|awk'{sub(/a+/,"<A>");print}'
Thisexampleusesthesub()functiontomakeachangetotheinputrecord.(sub()replacesthefirstinstanceofanytext
matchedbythefirstargumentwiththestringprovidedasthesecondargumentseeStringFunctions.)Here,theregexp
/a+/indicatesoneormoreacharacters,andthereplacementtextis<A>.
Theinputcontainsfouracharacters.awk(andPOSIX)regularexpressionsalwaysmatchtheleftmost,longest
sequenceofinputcharactersthatcanmatch.Thus,allfouracharactersarereplacedwith<A>inthisexample:
$echoaaaabcd|awk'{sub(/a+/,"<A>");print}'
|<A>bcd
Forsimplematch/nomatchtests,thisisnotsoimportant.Butwhendoingtextmatchingandsubstitutionswiththe
match(),sub(),gsub(),andgensub()functions,itisveryimportant.Understandingthisprincipleisalsoimportantfor
regexpbasedrecordandfieldsplitting(seeRecords,andalsoseeFieldSeparators).
Next:GNURegexpOperators,Previous:LeftmostLongest,Up:Regexp[Contents][Index]
3.6UsingDynamicRegexps
Therighthandsideofa~or!~operatorneednotbearegexpconstant(i.e.,astringofcharactersbetweenslashes).It
maybeanyexpression.Theexpressionisevaluatedandconvertedtoastringifnecessarythecontentsofthestringare
thenusedastheregexp.Aregexpcomputedinthiswayiscalledadynamicregexporacomputedregexp:
BEGIN{digits_regexp="[[:digit:]]+"}
$0~digits_regexp{print}
Thissetsdigits_regexptoaregexpthatdescribesoneormoredigits,andtestswhethertheinputrecordmatchesthis
regexp.
NOTE:Whenusingthe~and!~operators,beawarethatthereisadifferencebetweenaregexp
constantenclosedinslashesandastringconstantenclosedindoublequotes.Ifyouaregoingtouseastring
constant,youhavetounderstandthatthestringis,inessence,scannedtwice:thefirsttimewhenawkreads
yourprogram,andthesecondtimewhenitgoestomatchthestringonthelefthandsideoftheoperatorwith
thepatternontheright.Thisistrueofanystringvaluedexpression(suchasdigits_regexp,showninthe
previousexample),notjuststringconstants.
Whatdifferencedoesitmakeifthestringisscannedtwice?Theanswerhastodowithescapesequences,and
particularlywithbackslashes.Togetabackslashintoaregularexpressioninsideastring,youhavetotypetwo
backslashes.
Forexample,/\*/isaregexpconstantforaliteral*.Onlyonebackslashisneeded.Todothesamethingwithastring,
youhavetotype"\\*".Thefirstbackslashescapesthesecondonesothatthestringactuallycontainsthetwocharacters
\and*.
Giventhatyoucanusebothregexpandstringconstantstodescriberegularexpressions,whichshouldyouuse?The
answerisregexpconstants,forseveralreasons:
Stringconstantsaremorecomplicatedtowriteandmoredifficulttoread.Usingregexpconstantsmakesyour
58/479
29/7/2016
TheGNUAwkUsersGuide
programslesserrorprone.Notunderstandingthedifferencebetweenthetwokindsofconstantsisacommon
sourceoferrors.
Itismoreefficienttouseregexpconstants.awkcannotethatyouhavesuppliedaregexpandstoreitinternallyin
aformthatmakespatternmatchingmoreefficient.Whenusingastringconstant,awkmustfirstconvertthestring
intothisinternalformandthenperformthepatternmatching.
Usingregexpconstantsisbetterformitshowsclearlythatyouintendaregexpmatch.
Using\ninBracketExpressionsofDynamicRegexps
Someolderversionsofawkdonotallowthenewlinecharactertobeusedinsideabracketexpressionforadynamic
regexp:
$awk'$0~"[\t\n]"'
errorawk:newlineincharacterclass[
error]...
errorsourcelinenumber1
errorcontextis
error$0~"[>>>\t\n]"<<<
Butanewlineinaregexpconstantworkswithnoproblem:
$awk'$0~/[\t\n]/'
hereisasampleline
|hereisasampleline
Ctrld
gawkdoesnothavethisproblem,anditisntlikelytooccurofteninpractice,butitsworthnotingforfuturereference.
Next:Casesensitivity,Previous:ComputedRegexps,Up:Regexp[Contents][Index]
3.7gawkSpecificRegexpOperators
GNUsoftwarethatdealswithregularexpressionsprovidesanumberofadditionalregexpoperators.Theseoperatorsare
describedinthissectionandarespecifictogawktheyarenotavailableinotherawkimplementations.Mostofthe
additionaloperatorsdealwithwordmatching.Forourpurposes,awordisasequenceofoneormoreletters,digits,or
underscores(_):
\s
Matchesanywhitespacecharacter.Thinkofitasshorthandfor[[:space:]].
\S
Matchesanycharacterthatisnotwhitespace.Thinkofitasshorthandfor[^[:space:]].
\w
Matchesanywordconstituentcharacterthatis,itmatchesanyletter,digit,orunderscore.Thinkofitas
shorthandfor[[:alnum:]_].
\W
Matchesanycharacterthatisnotwordconstituent.Thinkofitasshorthandfor[^[:alnum:]_].
\<
Matchestheemptystringatthebeginningofaword.Forexample,/\<away/matchesawaybutnotstowaway.
\>
59/479
29/7/2016
TheGNUAwkUsersGuide
Matchestheemptystringattheendofaword.Forexample,/stow\>/matchesstowbutnotstowaway.
\y
Matchestheemptystringateitherthebeginningortheendofaword(i.e.,thewordboundary).Forexample,
\yballs?\ymatcheseither ballorballs,asaseparateword.
\B
Matchestheemptystringthatoccursbetweentwowordconstituentcharacters.Forexample,/\Brat\B/matches
crate,butitdoesnotmatchdirtyrat.\Bisessentiallytheoppositeof\y.
Therearetwootheroperatorsthatworkonbuffers.InEmacs,abufferis,naturally,anEmacsbuffer.OtherGNU
programs,includinggawk,considertheentirestringtomatchasthebuffer.Theoperatorsare:
\`
Matchestheemptystringatthebeginningofabuffer(string)
\'
Matchestheemptystringattheendofabuffer(string)
Becauseând$alwaysworkintermsofthebeginningandendofstrings,theseoperatorsdontaddanynew
capabilitiesforawk.TheyareprovidedforcompatibilitywithotherGNUsoftware.
InotherGNUsoftware,thewordboundaryoperatoris\b.However,thatconflictswiththeawklanguagesdefinition
of\basbackspace,so gawkusesadifferentletter.Analternativemethodwouldhavebeentorequiretwobackslashes
intheGNUoperators,butthiswasdeemedtooconfusing.Thecurrentmethodofusing\yfortheGNU\bappearsto
bethelesseroftwoevils.
Thevariouscommandlineoptions(seeOptions)controlhowgawkinterpretscharactersinregexps:
Nooptions
Inthedefaultcase,gawkprovidesallthefacilitiesofPOSIXregexpsandthepreviouslydescribedGNUregexp
operators.GNUregexpoperatorsdescribedinRegexpOperators.
posix
MatchonlyPOSIXregexpstheGNUoperatorsarenotspecial(e.g.,\wmatchesaliteralw).Interval
expressionsareallowed.
traditional
MatchtraditionalUnixawkregexps.TheGNUoperatorsarenotspecial,andintervalexpressionsarenot
available.BecauseBWKawksupportsthem,thePOSIXcharacterclasses([[:alnum:]],etc.)areavailable.
Charactersdescribedbyoctalandhexadecimalescapesequencesaretreatedliterally,eveniftheyrepresent
regexpmetacharacters.
reinterval
Allowintervalexpressionsinregexps,iftraditionalhasbeenprovided.Otherwise,intervalexpressionsare
availablebydefault.
Next:RegexpSummary,Previous:GNURegexpOperators,Up:Regexp[Contents][Index]
3.8CaseSensitivityinMatching
Caseisnormallysignificantinregularexpressions,bothwhenmatchingordinarycharacters(i.e.,notmetacharacters)
60/479
29/7/2016
TheGNUAwkUsersGuide
andinsidebracketexpressions.Thus,awinaregularexpressionmatchesonlyalowercasewandnotanuppercase
W.
Thesimplestwaytodoacaseindependentmatchistouseabracketexpressionforexample,[Ww].However,this
canbecumbersomeifyouneedtouseitoften,anditcanmaketheregularexpressionshardertoread.Therearetwo
alternativesthatyoumightprefer.
Onewaytoperformacaseinsensitivematchataparticularpointintheprogramistoconvertthedatatoasinglecase,
usingthetolower()ortoupper()builtinstringfunctions(whichwehaventdiscussedyetseeStringFunctions).For
example:
tolower($1)~/foo/{}
convertsthefirstfieldtolowercasebeforematchingagainstit.ThisworksinanyPOSIXcompliantawk.
Anothermethod,specifictogawk,istosetthevariableIGNORECASEtoanonzerovalue(seeBuiltinVariables).When
IGNORECASEisnotzero,allregexpandstringoperationsignorecase.
ChangingthevalueofIGNORECASEdynamicallycontrolsthecasesensitivityoftheprogramasitruns.Caseissignificant
bydefaultbecauseIGNORECASE(likemostvariables)isinitializedtozero:
x="aB"
if(x~/ab/)#thistestwillfail
IGNORECASE=1
if(x~/ab/)#nowitwillsucceed
Ingeneral,youcannotuseIGNORECASEtomakecertainrulescaseinsensitiveandotherrulescasesensitive,asthereisno
straightforwardwaytosetIGNORECASEjustforthepatternofaparticularrule.17Todothis,useeitherbracket
expressionsortolower().However,onethingyoucandowithIGNORECASEonlyisdynamicallyturncasesensitivityon
oroffforalltherulesatonce.
IGNORECASEcanbesetonthecommandlineorinaBEGINrule(seeOtherArgumentsalsoseeUsingBEGIN/END).
SettingIGNORECASEfromthecommandlineisawaytomakeaprogramcaseinsensitivewithouthavingtoeditit.
Inmultibytelocales,theequivalencesbetweenupperandlowercasecharactersaretestedbasedonthewidecharacter
valuesofthelocalescharacterset.Otherwise,thecharactersaretestedbasedontheISO88591(ISOLatin1)
characterset.Thischaractersetisasupersetofthetraditional128ASCIIcharacters,whichalsoprovidesanumberof
characterssuitableforusewithEuropeanlanguages.18
ThevalueofIGNORECASEhasnoeffectifgawkisincompatibilitymode(seeOptions).Caseisalwayssignificantin
compatibilitymode.
Previous:Casesensitivity,Up:Regexp[Contents][Index]
3.9Summary
Regularexpressionsdescribesetsofstringstobematched.Inawk,regularexpressionconstantsarewritten
enclosedbetweenslashes://.
Regexpconstantsmaybeusedstandaloneinpatternsandinconditionalexpressions,oraspartofmatching
expressionsusingthe~and!~operators.
Escapesequencesletyourepresentnonprintablecharactersandalsoletyourepresentregexpmetacharactersas
literalcharacterstobematched.
Regexpoperatorsprovidegrouping,alternation,andrepetition.
Bracketexpressionsgiveyouashorthandforspecifyingsetsofcharactersthatcanmatchataparticularpointina
regexp.Withinbracketexpressions,POSIXcharacterclassesletyouspecifycertaingroupsofcharactersina
localeindependentfashion.
Regularexpressionsmatchtheleftmostlongesttextinthestringbeingmatched.Thismattersforcaseswhereyou
61/479
29/7/2016
TheGNUAwkUsersGuide
needtoknowtheextentofthematch,suchasfortextsubstitutionandwhentherecordseparatorisaregexp.
Matchingexpressionsmayusedynamicregexps(i.e.,stringvaluestreatedasregularexpressions).
gawksIGNORECASEvariableletsyoucontrolthecasesensitivityofregexpmatching.Inotherawkversions,use
tolower()ortoupper().
Next:Printing,Previous:Regexp,Up:Top[Contents][Index]
4ReadingInputFiles
Inthetypicalawkprogram,awkreadsallinputeitherfromthestandardinput(bydefault,thisisthekeyboard,butoftenit
isapipefromanothercommand)orfromfileswhosenamesyouspecifyontheawkcommandline.Ifyouspecifyinput
files,awkreadstheminorder,processingallthedatafromonebeforegoingontothenext.Thenameofthecurrent
inputfilecanbefoundinthepredefinedvariableFILENAME(seeBuiltinVariables).
Theinputisreadinunitscalledrecords,andisprocessedbytherulesofyourprogramonerecordatatime.Bydefault,
eachrecordisoneline.Eachrecordisautomaticallysplitintochunkscalledfields.Thismakesitmoreconvenientfor
programstoworkonthepartsofarecord.
Onrareoccasions,youmayneedtousethegetlinecommand.Thegetlinecommandisvaluablebothbecauseitcando
explicitinputfromanynumberoffiles,andbecausethefilesusedwithitdonothavetobenamedontheawkcommand
line(seeGetline).
Records:
Controllinghowdataissplitintorecords.
Fields:
Anintroductiontofields.
NonconstantFields:
NonconstantFieldNumbers.
ChangingFields:
ChangingtheContentsofaField.
FieldSeparators:
Thefieldseparatorandhowtochangeit.
ConstantSize:
Readingconstantwidthdata.
SplittingByContent:
DefiningFieldsByContent
MultipleLine:
Readingmultilinerecords.
Getline:
Readingfilesunderexplicitprogramcontrolusingthegetlinefunction.
ReadTimeout:
Readinginputwithatimeout.
Commandlinedirectories: Whathappensifyouputadirectoryonthecommandline.
InputSummary:
Inputsummary.
InputExercises:
Exercises.
Next:Fields,Up:ReadingFiles[Contents][Index]
4.1HowInputIsSplitintoRecords
awkdividestheinputforyourprogramintorecordsandfields.Itkeepstrackofthenumberofrecordsthathavebeen
readsofarfromthecurrentinputfile.ThisvalueisstoredinapredefinedvariablecalledFNR,whichisresettozero
everytimeanewfileisstarted.Anotherpredefinedvariable,NR,recordsthetotalnumberofinputrecordsreadsofar
fromalldatafiles.Itstartsatzero,butisneverautomaticallyresettozero.
awksplitrecords: Howstandardawksplitsrecords.
gawksplitrecords: Howgawksplitsrecords.
Next:gawksplitrecords,Up:Records[Contents][Index]
4.1.1RecordSplittingwithStandardawk
Recordsareseparatedbyacharactercalledtherecordseparator.Bydefault,therecordseparatoristhenewline
62/479
29/7/2016
TheGNUAwkUsersGuide
character.Thisiswhyrecordsare,bydefault,singlelines.Touseadifferentcharacterfortherecordseparator,simply
assignthatcharactertothepredefinedvariableRS.
Likeanyothervariable,thevalueofRScanbechangedintheawkprogramwiththeassignmentoperator,=(see
AssignmentOps).Thenewrecordseparatorcharactershouldbeenclosedinquotationmarks,whichindicateastring
constant.Often,therighttimetodothisisatthebeginningofexecution,beforeanyinputisprocessed,sothatthevery
firstrecordisreadwiththeproperseparator.Todothis,usethespecialBEGINpattern(seeBEGIN/END).Forexample:
awk'BEGIN{RS="u"}
{print$0}'maillist
changesthevalueofRStou,beforereadinganyinput.Thenewvalueisastringwhosefirstcharacteristheletteru
asaresult,recordsareseparatedbytheletteru.Thentheinputfileisread,andthesecondruleintheawkprogram(the
actionwithnopattern)printseachrecord.Becauseeachprintstatementaddsanewlineattheendofitsoutput,thisawk
programcopiestheinputwitheachuchangedtoanewline.Herearetheresultsofrunningtheprogramonmaillist:
$awk'BEGIN{RS="u"}
>{print$0}'maillist
|Amelia5555553amelia.zodiac
|sq
|e@gmail.comF
|Anthony5553412anthony.assert
|ro@hotmail.comA
|Becky5557685becky.algebrar
|m@gmail.comA
|Bill5551675bill.drowning@hotmail.comA
|Broderick5550542broderick.aliq
|otiens@yahoo.comR
|Camilla5552912camilla.inf
|sar
|m@skynet.beR
|Fabi
|s5551234fabi
|s.
|ndevicesim
|s@
|cb.ed
|F
|J
|lie5556699j
|lie.perscr
|tabor@skeeve.comF
|Martin5556480martin.codicib
|s@hotmail.comA
|Sam
|el5553430sam
|el.lanceolis@sh
|.ed
|A
|JeanPa
|l5552127jeanpa
|l.campanor
|m@ny
|.ed
|R
|
NotethattheentryforthenameBillisnotsplit.Intheoriginaldatafile(seeSampleDataFiles),thelinelookslike
this:
Bill5551675bill.drowning@hotmail.comA
Itcontainsnou,sothereisnoreasontosplittherecord,unliketheothers,whicheachhaveoneormoreoccurrences
oftheu.Infact,thisrecordistreatedaspartofthepreviousrecordthenewlineseparatingthemintheoutputisthe
originalnewlineinthedatafile,nottheoneaddedbyawkwhenitprintedtherecord!
63/479
29/7/2016
TheGNUAwkUsersGuide
Anotherwaytochangetherecordseparatorisonthecommandline,usingthevariableassignmentfeature(seeOther
Arguments):
awk'{print$0}'RS="u"maillist
ThissetsRStoubeforeprocessingmaillist.
Usinganalphabeticcharactersuchasufortherecordseparatorishighlylikelytoproducestrangeresults.Usingan
unusualcharactersuchas/ismorelikelytoproducecorrectbehaviorinthemajorityofcases,butthereareno
guarantees.Themoralis:KnowYourData.
Whenusingregularcharactersastherecordseparator,thereisoneunusualcasethatoccurswhengawkisbeingfully
POSIXcompliant(seeOptions).Then,thefollowing(extreme)pipelineprintsasurprising1:
$echo|gawkposix'BEGIN{RS="a"};{printNF}'
|1
Thereisonefield,consistingofanewline.ThevalueofthebuiltinvariableNFisthenumberoffieldsinthecurrent
record.(Inthenormalcase,gawktreatsthenewlineaswhitespace,printing0astheresult.Mostotherversionsofawk
alsoactthisway.)
Reachingtheendofaninputfileterminatesthecurrentinputrecord,evenifthelastcharacterinthefileisnotthe
characterinRS.(d.c.)
Theemptystring""(astringwithoutanycharacters)hasaspecialmeaningasthevalueofRS.Itmeansthatrecordsare
separatedbyoneormoreblanklinesandnothingelse.SeeMultipleLine,formoredetails.
IfyouchangethevalueofRSinthemiddleofanawkrun,thenewvalueisusedtodelimitsubsequentrecords,butthe
recordcurrentlybeingprocessed,aswellasrecordsalreadyprocessed,arenotaffected.
Aftertheendoftherecordhasbeendetermined,gawksetsthevariableRTtothetextintheinputthatmatchedRS.
Previous:awksplitrecords,Up:Records[Contents][Index]
4.1.2RecordSplittingwithgawk
Whenusinggawk,thevalueofRSisnotlimitedtoaonecharacterstring.Itcanbeanyregularexpression(seeRegexp).
(c.e.)Ingeneral,eachrecordendsatthenextstringthatmatchestheregularexpressionthenextrecordstartsattheend
ofthematchingstring.Thisgeneralruleisactuallyatworkintheusualcase,whereRScontainsjustanewline:arecord
endsatthebeginningofthenextmatchingstring(thenextnewlineintheinput),andthefollowingrecordstartsjust
aftertheendofthisstring(atthefirstcharacterofthefollowingline).Thenewline,becauseitmatchesRS,isnotpartof
eitherrecord.
WhenRSisasinglecharacter,RTcontainsthesamesinglecharacter.However,whenRSisaregularexpression,RT
containstheactualinputtextthatmatchedtheregularexpression.
IftheinputfileendswithoutanytextmatchingRS,gawksetsRTtothenullstring.
Thefollowingexampleillustratesbothofthesefeatures.ItsetsRSequaltoaregularexpressionthatmatcheseithera
newlineoraseriesofoneormoreuppercaseletterswithoptionalleadingand/ortrailingwhitespace:
$echorecord1AAAArecord2BBBBrecord3|
>gawk'BEGIN{RS="\n|(*[[:upper:]]+*)"}
>{print"Record=",$0,"andRT=["RT"]"}'
|Record=record1andRT=[AAAA]
|Record=record2andRT=[BBBB]
|Record=record3andRT=[
|]
ThesquarebracketsdelineatethecontentsofRT,lettingyouseetheleadingandtrailingwhitespace.Thefinalvalueof
64/479
29/7/2016
TheGNUAwkUsersGuide
RTisanewline.SeeSimpleSed,foramoreusefulexampleofRSasaregexpandRT.
IfyousetRStoaregularexpressionthatallowsoptionaltrailingtext,suchasRS="abc(XYZ)?",itispossible,dueto
implementationconstraints,thatgawkmaymatchtheleadingpartoftheregularexpression,butnotthetrailingpart,
particularlyiftheinputtextthatcouldmatchthetrailingpartisfairlylong.gawkattemptstoavoidthisproblem,but
currently,theresnoguaranteethatthiswillneverhappen.
NOTE:Rememberthatinawk,theând$anchormetacharactersmatchthebeginningandendofa
string,andnotthebeginningandendofaline.Asaresult,somethinglikeRS="^[[:upper:]]"canonly
matchatthebeginningofafile.Thisisbecausegawkviewstheinputfileasonelongstringthathappensto
containnewlinecharacters.ItisthusbesttoavoidanchormetacharactersinthevalueofRS.
TheuseofRSasaregularexpressionandtheRTvariablearegawkextensionstheyarenotavailableincompatibility
mode(seeOptions).Incompatibilitymode,onlythefirstcharacterofthevalueofRSdeterminestheendoftherecord.
RS="\0"IsNotPortable
Therearetimeswhenyoumightwanttotreatanentiredatafileasasinglerecord.Theonlywaytomakethishappen
istogiveRSavaluethatyouknowdoesntoccurintheinputfile.Thisishardtodoinageneralway,suchthata
programalwaysworksforarbitraryinputfiles.
Youmightthinkthatfortextfiles,theNULcharacter,whichconsistsofacharacterwithallbitsequaltozero,isagood
valuetouseforRSinthiscase:
BEGIN{RS="\0"}#wholefilebecomesonerecord?
gawkinfactacceptsthis,andusestheNULcharacterfortherecordseparator.Thisworksforcertainspecialfiles,such
as/proc/environonGNU/Linuxsystems,wheretheNULcharacterisinfacttherecordseparator.However,thisusage
isnotportabletomostotherawkimplementations.
Almostallotherawkimplementations19storestringsinternallyasCstylestrings.CstringsusetheNULcharacteras
thestringterminator.Ineffect,thismeansthatRS="\0"isthesameasRS="".(d.c.)
IthappensthatrecentversionsofmawkcanusetheNULcharacterasarecordseparator.However,thisisaspecialcase:
mawkdoesnotallowembeddedNULcharactersinstrings.(Thismaychangeinafutureversionofmawk.)
SeeReadfileFunction,foraninterestingwaytoreadwholefiles.Ifyouareusinggawk,seeExtensionSample
Readfile,foranotheroption.
Next:NonconstantFields,Previous:Records,Up:ReadingFiles[Contents][Index]
4.2ExaminingFields
Whenawkreadsaninputrecord,therecordisautomaticallyparsedorseparatedbytheawkutilityintochunkscalled
fields.Bydefault,fieldsareseparatedbywhitespace,likewordsinaline.Whitespaceinawkmeansanystringofoneor
morespaces,TABs,ornewlines20othercharactersthatareconsideredwhitespacebyotherlanguages(suchas
formfeed,verticaltab,etc.)arenotconsideredwhitespacebyawk.
Thepurposeoffieldsistomakeitmoreconvenientforyoutorefertothesepiecesoftherecord.Youdonthavetouse
themyoucanoperateonthewholerecordifyouwantbutfieldsarewhatmakesimpleawkprogramssopowerful.
Youuseadollarsign($)torefertoafieldinanawkprogram,followedbythenumberofthefieldyouwant.Thus,$1
referstothefirstfield,$2tothesecond,andsoon.(UnlikeintheUnixshells,thefieldnumbersarenotlimitedtosingle
digits.$127isthe127thfieldintherecord.)Forexample,supposethefollowingisalineofinput:
Thisseemslikeaprettyniceexample.
65/479
29/7/2016
TheGNUAwkUsersGuide
Herethefirstfield,or$1,isThis,thesecondfield,or$2,isseems,andsoon.Notethatthelastfield,$7,is
example..Becausethereisnospacebetweentheeandthe.,theperiodisconsideredpartoftheseventhfield.
NFisapredefinedvariablewhosevalueisthenumberoffieldsinthecurrentrecord.awkautomaticallyupdatesthevalue
ofNFeachtimeitreadsarecord.Nomatterhowmanyfieldsthereare,thelastfieldinarecordcanberepresentedby
$NF.So,$NFisthesameas$7,whichisexample..Ifyoutrytoreferenceafieldbeyondthelastone(suchas$8when
therecordhasonlysevenfields),yougettheemptystring.(Ifusedinanumericoperation,yougetzero.)
Theuseof$0,whichlookslikeareferencetothezerothfield,isaspecialcase:itrepresentsthewholeinputrecord.
Useitwhenyouarenotinterestedinspecificfields.Herearesomemoreexamples:
$awk'$1~/li/{print$0}'maillist
Thisexampleprintseachrecordinthefilemaillistwhosefirstfieldcontainsthestringli.
Bycontrast,thefollowingexamplelooksforliintheentirerecordandprintsthefirstandlastfieldsforeach
matchinginputrecord:
$awk'/li/{print$1,$NF}'maillist
|AmeliaF
|BroderickR
|JulieF
|SamuelA
Next:ChangingFields,Previous:Fields,Up:ReadingFiles[Contents][Index]
4.3NonconstantFieldNumbers
Afieldnumberneednotbeaconstant.Anyexpressionintheawklanguagecanbeusedaftera$torefertoafield.The
valueoftheexpressionspecifiesthefieldnumber.Ifthevalueisastring,ratherthananumber,itisconvertedtoa
number.Considerthisexample:
awk'{print$NR}'
RecallthatNRisthenumberofrecordsreadsofar:oneinthefirstrecord,twointhesecond,andsoon.Sothisexample
printsthefirstfieldofthefirstrecord,thesecondfieldofthesecondrecord,andsoon.Forthetwentiethrecord,field
number20isprintedmostlikely,therecordhasfewerthan20fields,sothisprintsablankline.Hereisanother
exampleofusingexpressionsasfieldnumbers:
awk'{print$(2*2)}'maillist
awkevaluatestheexpression(2*2)andusesitsvalueasthenumberofthefieldtoprint.The*represents
multiplication,sotheexpression2*2evaluatestofour.Theparenthesesareusedsothatthemultiplicationisdone
beforethe$operationtheyarenecessarywheneverthereisabinaryoperator21inthefieldnumberexpression.This
example,then,printsthetypeofrelationship(thefourthfield)foreverylineofthefilemaillist.(Alloftheawk
operatorsarelisted,inorderofdecreasingprecedence,inPrecedence.)
Ifthefieldnumberyoucomputeiszero,yougettheentirerecord.Thus,$(22)hasthesamevalueas$0.Negative
fieldnumbersarenotallowedtryingtoreferenceoneusuallyterminatestheprogram.(ThePOSIXstandarddoesnot
definewhathappenswhenyoureferenceanegativefieldnumber.gawknoticesthisandterminatesyourprogram.Other
awkimplementationsmaybehavedifferently.)
AsmentionedinFields,awkstoresthecurrentrecordsnumberoffieldsinthebuiltinvariableNF(alsoseeBuiltin
Variables).Thus,theexpression$NFisnotaspecialfeatureitisthedirectconsequenceofevaluatingNFandusingits
valueasafieldnumber.
66/479
29/7/2016
TheGNUAwkUsersGuide
Next:FieldSeparators,Previous:NonconstantFields,Up:ReadingFiles[Contents][Index]
4.4ChangingtheContentsofaField
Thecontentsofafield,asseenbyawk,canbechangedwithinanawkprogramthischangeswhatawkperceivesasthe
currentinputrecord.(Theactualinputisuntouchedawknevermodifiestheinputfile.)Considerthefollowingexample
anditsoutput:
$awk'{nboxes=$3;$3=$310
>printnboxes,$3}'inventoryshipped
|2515
|3222
|2414
Theprogramfirstsavestheoriginalvalueoffieldthreeinthevariablenboxes.Thesignrepresentssubtraction,so
thisprogramreassignsfieldthree,$3,astheoriginalvalueoffieldthreeminusten:$310.(SeeArithmeticOps.)
Thenitprintstheoriginalandnewvaluesforfieldthree.(Someoneinthewarehousemadeaconsistentmistakewhile
inventoryingtheredboxes.)
Forthistowork,thetextin$3mustmakesenseasanumberthestringofcharactersmustbeconvertedtoanumberfor
thecomputertodoarithmeticonit.Thenumberresultingfromthesubtractionisconvertedbacktoastringofcharacters
thatthenbecomesfieldthree.SeeConversion.
Whenthevalueofafieldischanged(asperceivedbyawk),thetextoftheinputrecordisrecalculatedtocontainthenew
fieldwheretheoldonewas.Inotherwords,$0changestoreflectthealteredfield.Thus,thisprogramprintsacopyof
theinputfile,with10subtractedfromthesecondfieldofeachline:
$awk'{$2=$210;print$0}'inventoryshipped
|Jan32515115
|Feb53224226
|Mar52434228
Itisalsopossibletoassigncontentstofieldsthatareoutofrange.Forexample:
$awk'{$6=($5+$4+$3+$2)
>print$6}'inventoryshipped
|168
|297
|301
Wevejustcreated$6,whosevalueisthesumoffields$2,$3,$4,and$5.The+signrepresentsaddition.Forthefile
inventoryshipped,$6representsthetotalnumberofparcelsshippedforaparticularmonth.
Creatinganewfieldchangesawksinternalcopyofthecurrentinputrecord,whichisthevalueof$0.Thus,ifyoudo
print$0afteraddingafield,therecordprintedincludesthenewfield,withtheappropriatenumberoffieldseparators
betweenitandthepreviouslyexistingfields.
ThisrecomputationaffectsandisaffectedbyNF(thenumberoffieldsseeFields).Forexample,thevalueofNFissetto
thenumberofthehighestfieldyoucreate.Theexactformatof$0isalsoaffectedbyafeaturethathasnotbeen
discussedyet:theoutputfieldseparator,OFS,usedtoseparatethefields(seeOutputSeparators).
Note,however,thatmerelyreferencinganoutofrangefielddoesnotchangethevalueofeither$0orNF.Referencing
anoutofrangefieldonlyproducesanemptystring.Forexample:
if($(NF+1)!="")
print"can'thappen"
else
print"everythingisnormal"
67/479
29/7/2016
TheGNUAwkUsersGuide
shouldprinteverythingisnormal,becauseNF+1iscertaintobeoutofrange.(SeeIfStatement,formoreinformation
aboutawksifelsestatements.SeeTypingandComparison,formoreinformationaboutthe!=operator.)
Itisimportanttonotethatmakinganassignmenttoanexistingfieldchangesthevalueof$0butdoesnotchangethe
valueofNF,evenwhenyouassigntheemptystringtoafield.Forexample:
$echoabcd|awk'{OFS=":";$2=""
>print$0;printNF}'
|a::c:d
|4
Thefieldisstillthereitjusthasanemptyvalue,delimitedbythetwocolonsbetweenaandc.Thisexampleshows
whathappensifyoucreateanewfield:
$echoabcd|awk'{OFS=":";$2="";$6="new"
>print$0;printNF}'
|a::c:d::new
|6
Theinterveningfield,$5,iscreatedwithanemptyvalue(indicatedbythesecondpairofadjacentcolons),andNFis
updatedwiththevaluesix.
DecrementingNFthrowsawaythevaluesofthefieldsafterthenewvalueofNFandrecomputes$0.(d.c.)Hereisan
example:
$echoabcdef|awk'{print"NF=",NF;
>NF=3;print$0}'
|NF=6
|abc
CAUTION:Someversionsofawkdontrebuild$0whenNFisdecremented.
Finally,therearetimeswhenitisconvenienttoforceawktorebuildtheentirerecord,usingthecurrentvaluesofthe
fieldsandOFS.Todothis,usetheseeminglyinnocuousassignment:
$1=$1#forcerecordtobereconstituted
print$0#orwhateverelsewith$0
Thisforcesawktorebuildtherecord.Itdoeshelptoaddacomment,asweveshownhere.
Thereisaflipsidetotherelationshipbetween$0andthefields.Anyassignmentto$0causestherecordtobereparsed
intofieldsusingthecurrentvalueofFS.Thisalsoappliestoanybuiltinfunctionthatupdates$0,suchassub()and
gsub()(seeStringFunctions).
Understanding$0
Itisimportanttorememberthat$0isthefullrecord,exactlyasitwasreadfromtheinput.Thisincludesanyleadingor
trailingwhitespace,andtheexactwhitespace(orothercharacters)thatseparatesthefields.
ItisacommonerrortotrytochangethefieldseparatorsinarecordsimplybysettingFSandOFS,andthenexpectinga
plainprintorprint$0toprintthemodifiedrecord.
Butthisdoesnotwork,becausenothingwasdonetochangetherecorditself.Instead,youmustforcetherecordtobe
rebuilt,typicallywithastatementsuchas$1=$1,asdescribedearlier.
Next:ConstantSize,Previous:ChangingFields,Up:ReadingFiles[Contents][Index]
4.5SpecifyingHowFieldsAreSeparated
68/479
29/7/2016
TheGNUAwkUsersGuide
DefaultFieldSplitting:
Howfieldsarenormallyseparated.
RegexpFieldSplitting:
Usingregexpsasthefieldseparator.
SingleCharacterFields:
Makingeachcharacteraseparatefield.
CommandLineFieldSeparator: SettingFSfromthecommandline.
FullLineFields:
Makingthefulllinebeasinglefield.
FieldSplittingSummary:
Somefinalpointsandasummarytable.
Thefieldseparator,whichiseitherasinglecharacteroraregularexpression,controlsthewayawksplitsaninputrecord
intofields.awkscanstheinputrecordforcharactersequencesthatmatchtheseparatorthefieldsthemselvesarethetext
betweenthematches.
Intheexamplesthatfollow,weusethebulletsymbol()torepresentspacesintheoutput.Ifthefieldseparatorisoo,
thenthefollowingline:
moogoogaipan
issplitintothreefields:m,g,andgaipan.Notetheleadingspacesinthevaluesofthesecondandthirdfields.
ThefieldseparatorisrepresentedbythepredefinedvariableFS.Shellprogrammerstakenote:awkdoesnotusethename
IFSthatisusedbythePOSIXcompliantshells(suchastheUnixBourneshell,sh,orBash).
ThevalueofFScanbechangedintheawkprogramwiththeassignmentoperator,=(seeAssignmentOps).Often,the
righttimetodothisisatthebeginningofexecutionbeforeanyinputhasbeenprocessed,sothattheveryfirstrecordis
readwiththeproperseparator.Todothis,usethespecialBEGINpattern(seeBEGIN/END).Forexample,herewesetthe
valueofFStothestring",":
awk'BEGIN{FS=","};{print$2}'
Giventheinputline:
JohnQ.Smith,29OakSt.,Walamazoo,MI42139
thisawkprogramextractsandprintsthestring29OakSt..
Sometimestheinputdatacontainsseparatorcharactersthatdontseparatefieldsthewayyouthoughttheywould.For
instance,thepersonsnameintheexamplewejustusedmighthaveatitleorsuffixattached,suchas:
JohnQ.Smith,LXIX,29OakSt.,Walamazoo,MI42139
ThesameprogramwouldextractLXIXinsteadof29OakSt..Ifyouwereexpectingtheprogramtoprintthe
address,youwouldbesurprised.Themoralistochooseyourdatalayoutandseparatorcharacterscarefullytoprevent
suchproblems.(Ifthedataisnotinaformthatiseasytoprocess,perhapsyoucanmassageitfirstwithaseparateawk
program.)
Next:RegexpFieldSplitting,Up:FieldSeparators[Contents][Index]
4.5.1WhitespaceNormallySeparatesFields
Fieldsarenormallyseparatedbywhitespacesequences(spaces,TABs,andnewlines),notbysinglespaces.Twospaces
inarowdonotdelimitanemptyfield.ThedefaultvalueofthefieldseparatorFSisastringcontainingasinglespace,
"".Ifawkinterpretedthisvalueintheusualway,eachspacecharacterwouldseparatefields,sotwospacesinarow
wouldmakeanemptyfieldbetweenthem.ThereasonthisdoesnothappenisthatasinglespaceasthevalueofFSisa
specialcaseitistakentospecifythedefaultmannerofdelimitingfields.
IfFSisanyothersinglecharacter,suchas",",theneachoccurrenceofthatcharacterseparatestwofields.Two
consecutiveoccurrencesdelimitanemptyfield.Ifthecharacteroccursatthebeginningortheendoftheline,thattoo
delimitsanemptyfield.Thespacecharacteristheonlysinglecharacterthatdoesnotfollowtheserules.
69/479
29/7/2016
TheGNUAwkUsersGuide
Next:SingleCharacterFields,Previous:DefaultFieldSplitting,Up:FieldSeparators[Contents][Index]
4.5.2UsingRegularExpressionstoSeparateFields
TheprevioussubsectiondiscussedtheuseofsinglecharactersorsimplestringsasthevalueofFS.Moregenerally,the
valueofFSmaybeastringcontaininganyregularexpression.Inthiscase,eachmatchintherecordfortheregular
expressionseparatesfields.Forexample,theassignment:
FS=",\t"
makeseveryareaofaninputlinethatconsistsofacommafollowedbyaspaceandaTABintoafieldseparator.
Foralesstrivialexampleofaregularexpression,tryusingsinglespacestoseparatefieldsthewaysinglecommasare
used.FScanbesetto"[]"(leftbracket,space,rightbracket).Thisregularexpressionmatchesasinglespaceand
nothingelse(seeRegexp).
ThereisanimportantdifferencebetweenthetwocasesofFS=""(asinglespace)and FS="[\t\n]+"(aregular
expressionmatchingoneormorespaces,TABs,ornewlines).ForbothvaluesofFS,fieldsareseparatedbyruns
(multipleadjacentoccurrences)ofspaces,TABs,and/ornewlines.However,whenthevalueofFSis"",awkfirststrips
leadingandtrailingwhitespacefromtherecordandthendecideswherethefieldsare.Forexample,thefollowing
pipelineprintsb:
$echo'abcd'|awk'{print$2}'
|b
However,thispipelineprintsa(notetheextraspacesaroundeachletter):
$echo'abcd'|awk'BEGIN{FS="[\t\n]+"}
>{print$2}'
|a
Inthiscase,thefirstfieldisnull,orempty.
Thestrippingofleadingandtrailingwhitespacealsocomesintoplaywhenever$0isrecomputed.Forinstance,study
thispipeline:
$echo'abcd'|awk'{print;$2=$2;print}'
|abcd
|abcd
Thefirstprintstatementprintstherecordasitwasread,withleadingwhitespaceintact.Theassignmentto$2rebuilds
$0byconcatenating$1through$NFtogether,separatedbythevalueofOFS(whichisaspacebydefault).Becausethe
leadingwhitespacewasignoredwhenfinding$1,itisnotpartofthenew$0.Finally,thelastprintstatementprintsthe
new$0.
Thereisanadditionalsubtletytobeawareofwhenusingregularexpressionsforfieldsplitting.Itisnotwellspecified
inthePOSIXstandard,oranywhereelse,what^meanswhensplittingfields.Doesthe^matchonlyatthebeginning
oftheentirerecord?Oriseachfieldseparatoranewstring?Itturnsoutthatdifferentawkversionsanswerthisquestion
differently,andyoushouldnotrelyonanyspecificbehaviorinyourprograms.(d.c.)
Asapointofinformation,BWKawkallows^tomatchonlyatthebeginningoftherecord. gawkalsoworksthisway.
Forexample:
$echo'xxAAxxBxxC'|
>gawkF'(^x+)|(+)''{for(i=1;i<=NF;i++)
>printf">%s<\n",$i}'
|><
|>AA<
|>xxBxx<
|>C<
70/479
29/7/2016
TheGNUAwkUsersGuide
Next:CommandLineFieldSeparator,Previous:RegexpFieldSplitting,Up:FieldSeparators[Contents][Index]
4.5.3MakingEachCharacteraSeparateField
Therearetimeswhenyoumaywanttoexamineeachcharacterofarecordseparately.Thiscanbedoneingawkby
simplyassigningthenullstring("")toFS.(c.e.)Inthiscase,eachindividualcharacterintherecordbecomesaseparate
field.Forexample:
$echoab|gawk'BEGIN{FS=""}
>{
>for(i=1;i<=NF;i=i+1)
>print"Field",i,"is",$i
>}'
|Field1isa
|Field2is
|Field3isb
Traditionally,thebehaviorofFSequalto""wasnotdefined.Inthiscase,mostversionsofUnixawksimplytreatthe
entirerecordasonlyhavingonefield.(d.c.)Incompatibilitymode(seeOptions),ifFSisthenullstring,thengawkalso
behavesthisway.
Next:FullLineFields,Previous:SingleCharacterFields,Up:FieldSeparators[Contents][Index]
4.5.4SettingFSfromtheCommandLine
FScanbesetonthecommandline.UsetheFoptiontodoso.Forexample:
awkF,'program'inputfiles
setsFStothe,character.NoticethattheoptionusesanuppercaseFinsteadofalowercasef.Thelatteroption(f)
specifiesafilecontaininganawkprogram.
ThevalueusedfortheargumenttoFisprocessedinexactlythesamewayasassignmentstothepredefinedvariableFS.
Anyspecialcharactersinthefieldseparatormustbeescapedappropriately.Forexample,tousea\asthefield
separatoronthecommandline,youwouldhavetotype:
#sameasFS="\\"
awkF\\\\''files
Because\isusedforquotingintheshell,awkseesF\\.Thenawkprocessesthe\\forescapecharacters(see
EscapeSequences),finallyyieldingasingle\touseforthefieldseparator.
Asaspecialcase,incompatibilitymode(seeOptions),iftheargumenttoFist,thenFSissettotheTABcharacter.If
youtypeF\tattheshell,withoutanyquotes,the\getsdeleted,soawkfiguresthatyoureallywantyourfieldstobe
separatedwithTABsandnotts.UsevFS="t"orF"[t]"onthecommandlineifyoureallydowanttoseparate
yourfieldswithts.UseF'\t'whennotincompatibilitymodetospecifythatTABsseparatefields.
Asanexample,letsuseanawkprogramfilecallededu.awkthatcontainsthepattern/edu/andtheactionprint$1:
/edu/{print$1}
LetsalsosetFStobethecharacterandruntheprogramonthefilemaillist.Thefollowingcommandprintsalist
ofthenamesofthepeoplethatworkatorattendauniversity,andthefirstthreedigitsoftheirphonenumbers:
$awkFfedu.awkmaillist
|Fabius555
|Samuel555
|Jean
Notethethirdlineofoutput.Thethirdlineintheoriginalfilelookedlikethis:
71/479
29/7/2016
TheGNUAwkUsersGuide
JeanPaul5552127jeanpaul.campanorum@nyu.eduR
Theaspartofthepersonsnamewasusedasthefieldseparator,insteadoftheinthephonenumberthatwas
originallyintended.Thisdemonstrateswhyyouhavetobecarefulinchoosingyourfieldandrecordseparators.
PerhapsthemostcommonuseofasinglecharacterasthefieldseparatoroccurswhenprocessingtheUnixsystem
passwordfile.OnmanyUnixsystems,eachuserhasaseparateentryinthesystempasswordfile,withonelineperuser.
Theinformationintheselinesisseparatedbycolons.Thefirstfieldistheusersloginnameandthesecondistheusers
encryptedorshadowpassword.(Ashadowpasswordisindicatedbythepresenceofasinglexinthesecondfield.)A
passwordfileentrymightlooklikethis:
arnold:x:2076:10:ArnoldRobbins:/home/arnold:/bin/bash
Thefollowingprogramsearchesthesystempasswordfileandprintstheentriesforuserswhosefullnameisnot
indicated:
awkF:'$5==""'/etc/passwd
Next:FieldSplittingSummary,Previous:CommandLineFieldSeparator,Up:FieldSeparators[Contents][Index]
4.5.5MakingtheFullLineBeaSingleField
Occasionally,itsusefultotreatthewholeinputlineasasinglefield.Thiscanbedoneeasilyandportablysimplyby
settingFSto"\n"(anewline):22
awkF'\n''program'files
Whenyoudothis,$1isthesameas$0.
ChangingFSDoesNotAffecttheFields
AccordingtothePOSIXstandard,awkissupposedtobehaveasifeachrecordissplitintofieldsatthetimeitisread.
Inparticular,thismeansthatifyouchangethevalueofFSafterarecordisread,thevaluesofthefields(i.e.,howthey
weresplit)shouldreflecttheoldvalueofFS,notthenewone.
However,manyolderimplementationsofawkdonotworkthisway.Instead,theydefersplittingthefieldsuntilafield
isactuallyreferenced.ThefieldsaresplitusingthecurrentvalueofFS!(d.c.)Thisbehaviorcanbedifficultto
diagnose.Thefollowingexampleillustratesthedifferencebetweenthetwomethods:
sed1q/etc/passwd|awk'{FS=":";print$1}'
whichusuallyprints:
root
onanincorrectimplementationofawk,whilegawkprintsthefullfirstlineofthefile,somethinglike:
root:x:0:0:Root:/:
(Thesed23commandprintsjustthefirstlineof/etc/passwd.)
Previous:FullLineFields,Up:FieldSeparators[Contents][Index]
4.5.6FieldSplittingSummary
ItisimportanttorememberthatwhenyouassignastringconstantasthevalueofFS,itundergoesnormalawkstring
processing.Forexample,withUnixawkandgawk,theassignmentFS="\.."assignsthecharacterstring".."toFS
72/479
29/7/2016
TheGNUAwkUsersGuide
(thebackslashisstripped).Thiscreatesaregexpmeaningfieldsareseparatedbyoccurrencesofanytwocharacters.If
insteadyouwantfieldstobeseparatedbyaliteralperiodfollowedbyanysinglecharacter,useFS="\\..".
Thefollowinglistsummarizeshowfieldsaresplit,basedonthevalueofFS(==meansisequalto):
FS==""
Fieldsareseparatedbyrunsofwhitespace.Leadingandtrailingwhitespaceareignored.Thisisthedefault.
FS==anyothersinglecharacter
Fieldsareseparatedbyeachoccurrenceofthecharacter.Multiplesuccessiveoccurrencesdelimitemptyfields,as
doleadingandtrailingoccurrences.Thecharactercanevenbearegexpmetacharacteritdoesnotneedtobe
escaped.
FS==regexp
Fieldsareseparatedbyoccurrencesofcharactersthatmatchregexp.Leadingandtrailingmatchesofregexp
delimitemptyfields.
FS==""
Eachindividualcharacterintherecordbecomesaseparatefield.(Thisisacommonextensionitisnotspecified
bythePOSIXstandard.)
FSandIGNORECASE
TheIGNORECASEvariable(seeUsermodified)affectsfieldsplittingonlywhenthevalueofFSisaregexp.Ithasno
effectwhenFSisasinglecharacter,evenifthatcharacterisaletter.Thus,inthefollowingcode:
FS="c"
IGNORECASE=1
$0="aCa"
print$1
TheoutputisaCa.Ifyoureallywanttosplitfieldsonanalphabeticcharacterwhileignoringcase,usearegexpthat
willdoitforyou(e.g.,FS="[c]").Inthiscase,IGNORECASEwilltakeeffect.
Next:SplittingByContent,Previous:FieldSeparators,Up:ReadingFiles[Contents][Index]
4.6ReadingFixedWidthData
Thissectiondiscussesanadvancedfeatureofgawk.Ifyouareanoviceawkuser,youmightwanttoskipitonthefirst
reading.
gawkprovidesafacilityfordealingwithfixedwidthfieldswithnodistinctivefieldseparator.Forexample,dataofthis
naturearisesintheinputforoldFortranprogramswherenumbersareruntogether,orintheoutputofprogramsthatdid
notanticipatetheuseoftheiroutputasinputforotherprograms.
Anexampleofthelatterisatablewhereallthecolumnsarelinedupbytheuseofavariablenumberofspacesand
emptyfieldsarejustspaces.Clearly,awksnormalfieldsplittingbasedonFSdoesnotworkwellinthiscase.Althougha
portableawkprogramcanuseaseriesofsubstr()callson$0(seeStringFunctions),thisisawkwardandinefficientfor
alargenumberoffields.
Thesplittingofaninputrecordintofixedwidthfieldsisspecifiedbyassigningastringcontainingspaceseparated
numberstothebuiltinvariableFIELDWIDTHS.Eachnumberspecifiesthewidthofthefield,includingcolumnsbetween
fields.Ifyouwanttoignorethecolumnsbetweenfields,youcanspecifythewidthasaseparatefieldthatis
subsequentlyignored.Itisafatalerrortosupplyafieldwidththathasanegativevalue.Thefollowingdataistheoutput
73/479
29/7/2016
TheGNUAwkUsersGuide
oftheUnixwutility.ItisusefultoillustratetheuseofFIELDWIDTHS:
10:06pmup21days,14:04,23users
UserttyloginidleJCPUPCPUwhat
hzuottyV08:58pm95vip24.tex
hzangttyV36:37pm50csh
eklyettyV59:53pm71emthes.tex
dporteinttyV68:17pm1:47csh
gierdttyD310:00pm1elm
davettyD49:47pm44w
brentttyp026Jun914:4626:464:41bash
davettyq426Jun9115days4646wnewmail
Thefollowingprogramtakesthisinput,convertstheidletimetonumberofseconds,andprintsoutthefirsttwofields
andthecalculatedidletime:
BEGIN{FIELDWIDTHS="961067735"}
NR>2{
idle=$4
sub(/^+/,"",idle)#stripleadingspaces
if(idle=="")
idle=0
if(idle~/:/){
split(idle,t,":")
idle=t[1]*60+t[2]
}
if(idle~/days/)
idle*=24*60*60
print$1,$2,idle
}
NOTE:Theprecedingprogramusesanumberofawkfeaturesthathaventbeenintroducedyet.
Runningtheprogramonthedataproducesthefollowingresults:
hzuottyV00
hzangttyV350
eklyettyV50
dporteinttyV6107
gierdttyD31
davettyD40
brentttyp0286
davettyq41296000
Another(possiblymorepractical)exampleoffixedwidthinputdataistheinputfromadeckofballotingcards.Insome
partsoftheUnitedStates,votersmarktheirchoicesbypunchingholesincomputercards.Thesecardsarethen
processedtocountthevotesforanyparticularcandidateoronanyparticularissue.Becauseavotermaychoosenotto
voteonsomeissue,anycolumnonthecardmaybeempty.Anawkprogramforprocessingsuchdatacouldusethe
FIELDWIDTHSfeaturetosimplifyreadingthedata.(Ofcourse,gettinggawktorunonasystemwithcardreadersisanother
story!)
AssigningavaluetoFScausesgawktouseFSforfieldsplittingagain.UseFS=FStomakethishappen,without
havingtoknowthecurrentvalueofFS.Inordertotellwhichkindoffieldsplittingisineffect,usePROCINFO["FS"](see
Autoset).Thevalueis"FS"ifregularfieldsplittingisbeingused,or"FIELDWIDTHS"iffixedwidthfieldsplittingis
beingused:
if(PROCINFO["FS"]=="FS")
regularfieldsplitting
elseif(PROCINFO["FS"]=="FIELDWIDTHS")
fixedwidthfieldsplitting
else
contentbasedfieldsplitting(seenextsection)
ThisinformationisusefulwhenwritingafunctionthatneedstotemporarilychangeFSorFIELDWIDTHS,readsome
74/479
29/7/2016
TheGNUAwkUsersGuide
records,andthenrestoretheoriginalsettings(seePasswdFunctions,foranexampleofsuchafunction).
Next:MultipleLine,Previous:ConstantSize,Up:ReadingFiles[Contents][Index]
4.7DefiningFieldsbyContent
Thissectiondiscussesanadvancedfeatureofgawk.Ifyouareanoviceawkuser,youmightwanttoskipitonthefirst
reading.
Normally,whenusingFS,gawkdefinesthefieldsasthepartsoftherecordthatoccurinbetweeneachfieldseparator.In
otherwords,FSdefineswhatafieldisnot,insteadofwhatafieldis.However,therearetimeswhenyoureallywantto
definethefieldsbywhattheyare,andnotbywhattheyarenot.
Themostnotorioussuchcaseissocalledcommaseparatedvalues(CSV)data.Manyspreadsheetprograms,for
example,canexporttheirdataintotextfiles,whereeachrecordisterminatedwithanewline,andfieldsareseparatedby
commas.Ifcommasonlyseparatedthedata,therewouldntbeanissue.Theproblemcomeswhenoneofthefields
containsanembeddedcomma.Insuchcases,mostprogramsembedthefieldindoublequotes.24So,wemighthave
datalikethis:
Robbins,Arnold,"1234APrettyStreet,NE",MyTown,MyState,123456789,USA
TheFPATvariableoffersasolutionforcaseslikethis.ThevalueofFPATshouldbeastringthatprovidesaregular
expression.Thisregularexpressiondescribesthecontentsofeachfield.
InthecaseofCSVdataaspresentedhere,eachfieldiseitheranythingthatisnotacomma,oradoublequote,
anythingthatisnotadoublequote,andaclosingdoublequote.Ifwrittenasaregularexpressionconstant(see
Regexp),wewouldhave/([^,]+)|("[^"]+")/.Writingthisasastringrequiresustoescapethedoublequotes,leading
to:
FPAT="([^,]+)|(\"[^\"]+\")"
Puttingthistouse,hereisasimpleprogramtoparsethedata:
BEGIN{
FPAT="([^,]+)|(\"[^\"]+\")"
}
{
print"NF=",NF
for(i=1;i<=NF;i++){
printf("$%d=<%s>\n",i,$i)
}
}
Whenrun,wegetthefollowing:
$gawkfsimplecsv.awkaddresses.csv
NF=7
$1=<Robbins>
$2=<Arnold>
$3=<"1234APrettyStreet,NE">
$4=<MyTown>
$5=<MyState>
$6=<123456789>
$7=<USA>
Notetheembeddedcommainthevalueof$3.
AstraightforwardimprovementwhenprocessingCSVdataofthissortwouldbetoremovethequoteswhentheyoccur,
withsomethinglikethis:
75/479
29/7/2016
TheGNUAwkUsersGuide
if(substr($i,1,1)=="\""){
len=length($i)
$i=substr($i,2,len2)#Gettextwithinthetwoquotes
}
AswithFS,theIGNORECASEvariable(seeUsermodified)affectsfieldsplittingwithFPAT.
AssigningavaluetoFPAToverridesfieldsplittingwithFSandwithFIELDWIDTHS.SimilartoFIELDWIDTHS,thevalueof
PROCINFO["FS"]willbe"FPAT"ifcontentbasedfieldsplittingisbeingused.
NOTE:SomeprogramsexportCSVdatathatcontainsembeddednewlinesbetweenthedoublequotes.
gawkprovidesnowaytodealwiththis.EventhoughaformalspecificationforCSVdataexists,thereisnt
muchmoretobedonetheFPATmechanismprovidesanelegantsolutionforthemajorityofcases,andthe
gawkdevelopersaresatisfiedwiththat.
Aswritten,theregexpusedforFPATrequiresthateachfieldcontainatleastonecharacter.Astraightforward
modification(changingthefirst+to*)allowsfieldstobeempty:
FPAT="([^,]*)|(\"[^\"]+\")"
Finally,thepatsplit()functionmakesthesamefunctionalityavailableforsplittingregularstrings(seeString
Functions).
Torecap,gawkprovidesthreeindependentmethodstosplitinputrecordsintofields.Themechanismusedisbasedon
whichofthethreevariablesFS,FIELDWIDTHS,orFPATwaslastassignedto.
Next:Getline,Previous:SplittingByContent,Up:ReadingFiles[Contents][Index]
4.8MultipleLineRecords
Insomedatabases,asinglelinecannotconvenientlyholdalltheinformationinoneentry.Insuchcases,youcanuse
multilinerecords.Thefirststepindoingthisistochooseyourdataformat.
Onetechniqueistouseanunusualcharacterorstringtoseparaterecords.Forexample,youcouldusetheformfeed
character(written\finawk,asinC)toseparatethem,makingeachrecordapageofthefile.Todothis,justsetthe
variableRSto"\f"(astringcontainingtheformfeedcharacter).Anyothercharactercouldequallywellbeused,aslong
asitwontbepartofthedatainarecord.
Anothertechniqueistohaveblanklinesseparaterecords.Byaspecialdispensation,anemptystringasthevalueofRS
indicatesthatrecordsareseparatedbyoneormoreblanklines.WhenRSissettotheemptystring,eachrecordalways
endsatthefirstblanklineencountered.Thenextrecorddoesntstartuntilthefirstnonblanklinethatfollows.Nomatter
howmanyblanklinesappearinarow,theyallactasonerecordseparator.(Blanklinesmustbecompletelyemptylines
thatcontainonlywhitespacedonotcount.)
YoucanachievethesameeffectasRS=""byassigningthestring"\n\n+"toRS.Thisregexpmatchesthenewlineat
theendoftherecordandoneormoreblanklinesaftertherecord.Inaddition,aregularexpressionalwaysmatchesthe
longestpossiblesequencewhenthereisachoice(seeLeftmostLongest).So,thenextrecorddoesntstartuntilthefirst
nonblanklinethatfollowsnomatterhowmanyblanklinesappearinarow,theyareconsideredonerecordseparator.
However,thereisanimportantdifferencebetweenRS=""andRS="\n\n+".Inthefirstcase,leadingnewlinesin
theinputdatafileareignored,andifafileendswithoutextrablanklinesafterthelastrecord,thefinalnewlineis
removedfromtherecord.Inthesecondcase,thisspecialprocessingisnotdone.(d.c.)
Nowthattheinputisseparatedintorecords,thesecondstepistoseparatethefieldsintherecords.Onewaytodothisis
todivideeachofthelinesintofieldsinthenormalmanner.Thishappensbydefaultastheresultofaspecialfeature.
WhenRSissettotheemptystringandFSissettoasinglecharacter,thenewlinecharacteralwaysactsasafield
separator.ThisisinadditiontowhateverfieldseparationsresultfromFS.25
76/479
29/7/2016
TheGNUAwkUsersGuide
Theoriginalmotivationforthisspecialexceptionwasprobablytoprovideusefulbehaviorinthedefaultcase(i.e.,FSis
equalto"").Thisfeaturecanbeaproblemifyoureallydontwantthenewlinecharactertoseparatefields,because
thereisnowaytopreventit.However,youcanworkaroundthisbyusingthesplit()functiontobreakuptherecord
manually(seeStringFunctions).Ifyouhaveasinglecharacterfieldseparator,youcanworkaroundthespecialfeature
inadifferentway,bymakingFSintoaregexpforthatsinglecharacter.Forexample,ifthefieldseparatorisapercent
character,insteadofFS="%",useFS="[%]".
Anotherwaytoseparatefieldsistoputeachfieldonaseparateline:todothis,justsetthevariableFStothestring"\n".
(Thissinglecharacterseparatormatchesasinglenewline.)Apracticalexampleofadatafileorganizedthiswaymight
beamailinglist,whereblanklinesseparatetheentries.Consideramailinglistinafilenamedaddresses,whichlooks
likethis:
JaneDoe
123MainStreet
Anywhere,SE123456789
JohnSmith
456TreelinedAvenue
Smallville,MW987654321
Asimpleprogramtoprocessthisfileisasfollows:
#addrs.awksimplemailinglistprogram
#Recordsareseparatedbyblanklines.
#Eachlineisonefield.
BEGIN{RS="";FS="\n"}
{
print"Nameis:",$1
print"Addressis:",$2
print"CityandStateare:",$3
print""
}
Runningtheprogramproducesthefollowingoutput:
$awkfaddrs.awkaddresses
|Nameis:JaneDoe
|Addressis:123MainStreet
|CityandStateare:Anywhere,SE123456789
|
|Nameis:JohnSmith
|Addressis:456TreelinedAvenue
|CityandStateare:Smallville,MW987654321
|
SeeLabelsProgram,foramorerealisticprogramdealingwithaddresslists.Thefollowinglistsummarizeshowrecords
aresplit,basedonthevalueofRS:
RS=="\n"
Recordsareseparatedbythenewlinecharacter(\n).Ineffect,everylineinthedatafileisaseparaterecord,
includingblanklines.Thisisthedefault.
RS==anysinglecharacter
Recordsareseparatedbyeachoccurrenceofthecharacter.Multiplesuccessiveoccurrencesdelimitempty
records.
RS==""
77/479
29/7/2016
TheGNUAwkUsersGuide
Recordsareseparatedbyrunsofblanklines.WhenFSisasinglecharacter,thenthenewlinecharacteralways
servesasafieldseparator,inadditiontowhatevervalueFSmayhave.Leadingandtrailingnewlinesinafileare
ignored.
RS==regexp
Recordsareseparatedbyoccurrencesofcharactersthatmatchregexp.Leadingandtrailingmatchesofregexp
delimitemptyrecords.(ThisisagawkextensionitisnotspecifiedbythePOSIXstandard.)
Ifnotincompatibilitymode(seeOptions),gawksetsRTtotheinputtextthatmatchedthevaluespecifiedbyRS.Butif
theinputfileendedwithoutanytextthatmatchesRS,thengawksetsRTtothenullstring.
Next:ReadTimeout,Previous:MultipleLine,Up:ReadingFiles[Contents][Index]
4.9ExplicitInputwithgetline
Sofarwehavebeengettingourinputdatafromawksmaininputstreameitherthestandardinput(usuallyyour
keyboard,sometimestheoutputfromanotherprogram)orthefilesspecifiedonthecommandline.Theawklanguage
hasaspecialbuiltincommandcalledgetlinethatcanbeusedtoreadinputunderyourexplicitcontrol.
Thegetlinecommandisusedinseveraldifferentwaysandshouldnotbeusedbybeginners.Theexamplesthatfollow
theexplanationofthegetlinecommandincludematerialthathasnotbeencoveredyet.Therefore,comebackandstudy
thegetlinecommandafteryouhavereviewedtherestofthisWebpageandhaveagoodknowledgeofhowawkworks.
Thegetlinecommandreturns1ifitfindsarecordand0ifitencounterstheendofthefile.Ifthereissomeerrorin
gettingarecord,suchasafilethatcannotbeopened,thengetlinereturns1.Inthiscase,gawksetsthevariableERRNOto
astringdescribingtheerrorthatoccurred.
Inthefollowingexamples,commandstandsforastringvaluethatrepresentsashellcommand.
NOTE:Whensandboxisspecified(seeOptions),readinglinesfromfiles,pipes,andcoprocessesis
disabled.
PlainGetline:
Usinggetlinewithnoarguments.
Getline/Variable:
Usinggetlineintoavariable.
Getline/File:
Usinggetlinefromafile.
Getline/Variable/File:
Usinggetlineintoavariablefromafile.
Getline/Pipe:
Usinggetlinefromapipe.
Getline/Variable/Pipe:
Usinggetlineintoavariablefromapipe.
Getline/Coprocess:
Usinggetlinefromacoprocess.
Getline/Variable/Coprocess: Usinggetlineintoavariablefromacoprocess.
GetlineNotes:
Importantthingstoknowaboutgetline.
GetlineSummary:
SummaryofgetlineVariants.
Next:Getline/Variable,Up:Getline[Contents][Index]
4.9.1UsinggetlinewithNoArguments
Thegetlinecommandcanbeusedwithoutargumentstoreadinputfromthecurrentinputfile.Allitdoesinthiscaseis
readthenextinputrecordandsplititupintofields.Thisisusefulifyouvefinishedprocessingthecurrentrecord,but
wanttodosomespecialprocessingonthenextrecordrightnow.Forexample:
#Removetextbetween/*and*/,inclusive
{
if((i=index($0,"/*"))!=0){
out=substr($0,1,i1)#leadingpartofthestring
78/479
29/7/2016
TheGNUAwkUsersGuide
rest=substr($0,i+2)#...*/...
j=index(rest,"*/")#is*/intrailingpart?
if(j>0){
rest=substr(rest,j+2)#removecomment
}else{
while(j==0){
#getmoretext
if(getline<=0){
print("unexpectedEOForerror:",ERRNO)>"/dev/stderr"
exit
}
#buildupthelineusingstringconcatenation
rest=rest$0
j=index(rest,"*/")#is*/intrailingpart?
if(j!=0){
rest=substr(rest,j+2)
break
}
}
}
#builduptheoutputlineusingstringconcatenation
$0=outrest
}
print$0
}
ThisawkprogramdeletesCstylecomments(/**/)fromtheinput.Itusesanumberoffeatureswehaventcovered
yet,includingstringconcatenation(seeConcatenation)andtheindex()andsubstr()builtinfunctions(seeString
Functions).Byreplacingtheprint$0withotherstatements,youcouldperformmorecomplicatedprocessingonthe
decommentedinput,suchassearchingformatchesofaregularexpression.(Thisprogramhasasubtleproblemitdoes
notworkifonecommentendsandanotherbeginsonthesameline.)
ThisformofthegetlinecommandsetsNF,NR,FNR,RT,andthevalueof$0.
NOTE:Thenewvalueof$0isusedtotestthepatternsofanysubsequentrules.Theoriginalvalueof$0
thattriggeredtherulethatexecutedgetlineislost.Bycontrast,thenextstatementreadsanewrecordbut
immediatelybeginsprocessingitnormally,startingwiththefirstruleintheprogram.SeeNextStatement.
Next:Getline/File,Previous:PlainGetline,Up:Getline[Contents][Index]
4.9.2UsinggetlineintoaVariable
Youcanusegetlinevartoreadthenextrecordfromawksinputintothevariablevar.Nootherprocessingisdone.
Forexample,supposethenextlineisacommentoraspecialstring,andyouwanttoreaditwithouttriggeringanyrules.
Thisformofgetlineallowsyoutoreadthatlineandstoreitinavariablesothatthemainreadalineandcheckeach
ruleloopofawkneverseesit.Thefollowingexampleswapseverytwolinesofinput:
{
if((getlinetmp)>0){
printtmp
print$0
}else
print$0
}
Ittakesthefollowinglist:
wan
tew
free
phore
andproducestheseresults:
79/479
29/7/2016
TheGNUAwkUsersGuide
tew
wan
phore
free
ThegetlinecommandusedinthiswaysetsonlythevariablesNR,FNR,andRT(and,ofcourse,var).Therecordisnot
splitintofields,sothevaluesofthefields(including$0)andthevalueofNFdonotchange.
Next:Getline/Variable/File,Previous:Getline/Variable,Up:Getline[Contents][Index]
4.9.3UsinggetlinefromaFile
Usegetline<filetoreadthenextrecordfromfile.Here,fileisastringvaluedexpressionthatspecifiesthefile
name.<fileiscalledaredirectionbecauseitdirectsinputtocomefromadifferentplace.Forexample,thefollowing
programreadsitsinputrecordfromthefilesecondary.inputwhenitencountersafirstfieldwithavalueequalto10in
thecurrentinputfile:
{
if($1==10){
getline<"secondary.input"
print
}else
print
}
Becausethemaininputstreamisnotused,thevaluesofNRandFNRarenotchanged.However,therecorditreadsissplit
intofieldsinthenormalmanner,sothevaluesof$0andtheotherfieldsarechanged,resultinginanewvalueofNF.RT
isalsoset.
AccordingtoPOSIX,getline<expressionisambiguousifexpressioncontainsunparenthesizedoperatorsotherthan
$forexample,getline<dir"/"fileisambiguousbecausetheconcatenationoperator(notdiscussedyetsee
Concatenation)isnotparenthesized.Youshouldwriteitasgetline<(dir"/"file)ifyouwantyourprogramtobe
portabletoallawkimplementations.
Next:Getline/Pipe,Previous:Getline/File,Up:Getline[Contents][Index]
4.9.4UsinggetlineintoaVariablefromaFile
Usegetlinevar<filetoreadinputfromthefilefile,andputitinthevariablevar.Asearlier,fileisastringvalued
expressionthatspecifiesthefilefromwhichtoread.
Inthisversionofgetline,noneofthepredefinedvariablesarechangedandtherecordisnotsplitintofields.Theonly
variablechangedisvar.26Forexample,thefollowingprogramcopiesalltheinputfilestotheoutput,exceptforrecords
thatsay@includefilename.Sucharecordisreplacedbythecontentsofthefilefilename:
{
if(NF==2&&$1=="@include"){
while((getlineline<$2)>0)
printline
close($2)
}else
print
}
Noteherehowthenameoftheextrainputfileisnotbuiltintotheprogramitistakendirectlyfromthedata,
specificallyfromthesecondfieldonthe@includeline.
Theclose()functioniscalledtoensurethatiftwoidentical@includelinesappearintheinput,theentirespecifiedfile
isincludedtwice.SeeCloseFilesAndPipes.
80/479
29/7/2016
TheGNUAwkUsersGuide
Onedeficiencyofthisprogramisthatitdoesnotprocessnested@includestatements(i.e.,@includestatementsin
includedfiles)thewayatruemacropreprocessorwould.SeeIgawkProgram,foraprogramthatdoeshandlenested
@includestatements.
Next:Getline/Variable/Pipe,Previous:Getline/Variable/File,Up:Getline[Contents][Index]
4.9.5UsinggetlinefromaPipe
Omnisciencehasmuchtorecommendit.Failingthat,attentiontodetailswouldbeuseful.
BrianKernighan
Theoutputofacommandcanalsobepipedintogetline,usingcommand|getline.Inthiscase,thestringcommandis
runasashellcommandanditsoutputispipedintoawktobeusedasinput.Thisformofgetlinereadsonerecordata
timefromthepipe.Forexample,thefollowingprogramcopiesitsinputtoitsoutput,exceptforlinesthatbeginwith
@execute,whicharereplacedbytheoutputproducedbyrunningtherestofthelineasashellcommand:
{
if($1=="@execute"){
tmp=substr($0,10)#Remove"@execute"
while((tmp|getline)>0)
print
close(tmp)
}else
print
}
Theclose()functioniscalledtoensurethatiftwoidentical@executelinesappearintheinput,thecommandisrunfor
eachone.SeeCloseFilesAndPipes.Giventheinput:
foo
bar
baz
@executewho
bletch
theprogrammightproduce:
foo
bar
baz
arnoldttyv0Jul1314:22
miriamttyp0Jul1314:23(murphy:0)
billttyp1Jul1314:23(murphy:0)
bletch
Noticethatthisprogramranthecommandwhoandprintedtheresult.(Ifyoutrythisprogramyourself,youwillof
coursegetdifferentresults,dependinguponwhoisloggedinonyoursystem.)
Thisvariationofgetlinesplitstherecordintofields,setsthevalueofNF,andrecomputesthevalueof$0.Thevaluesof
NRandFNRarenotchanged.RTisset.
AccordingtoPOSIX,expression|getlineisambiguousifexpressioncontainsunparenthesizedoperatorsotherthan
$forexample,"echo""date"|getlineisambiguousbecausetheconcatenationoperatorisnotparenthesized.
Youshouldwriteitas("echo""date")|getlineifyouwantyourprogramtobeportabletoallawk
implementations.
NOTE:Unfortunately,gawkhasnotbeenconsistentinitstreatmentofaconstructlike"echo""date"|
getline.Mostversions,includingthecurrentversion,treatitatas("echo""date")|getline.(Thisis
alsohowBWKawkbehaves.)Someversionsinsteadtreatitas"echo"("date"|getline).(Thisishow
mawkbehaves.)Inshort,alwaysuseexplicitparentheses,andthenyouwonthavetoworry.
81/479
29/7/2016
TheGNUAwkUsersGuide
Next:Getline/Coprocess,Previous:Getline/Pipe,Up:Getline[Contents][Index]
4.9.6UsinggetlineintoaVariablefromaPipe
Whenyouusecommand|getlinevar,theoutputofcommandissentthroughapipetogetlineandintothevariable
var.Forexample,thefollowingprogramreadsthecurrentdateandtimeintothevariablecurrent_time,usingthedate
utility,andthenprintsit:
BEGIN{
"date"|getlinecurrent_time
close("date")
print"Reportprintedon"current_time
}
Inthisversionofgetline,noneofthepredefinedvariablesarechangedandtherecordisnotsplitintofields.However,
RTisset.
Next:Getline/Variable/Coprocess,Previous:Getline/Variable/Pipe,Up:Getline[Contents][Index]
4.9.7UsinggetlinefromaCoprocess
Readinginputintogetlinefromapipeisaonewayoperation.Thecommandthatisstartedwithcommand|getline
onlysendsdatatoyourawkprogram.
Onoccasion,youmightwanttosenddatatoanotherprogramforprocessingandthenreadtheresultsback.gawkallows
youtostartacoprocess,withwhichtwowaycommunicationsarepossible.Thisisdonewiththe|&operator.
Typically,youwritedatatothecoprocessfirstandthenreadtheresultsback,asshowninthefollowing:
print"somequery"|&"db_server"
"db_server"|&getline
whichsendsaquerytodb_serverandthenreadstheresults.
ThevaluesofNRandFNRarenotchanged,becausethemaininputstreamisnotused.However,therecordissplitinto
fieldsinthenormalmanner,thuschangingthevaluesof$0,oftheotherfields,andofNFandRT.
Coprocessesareanadvancedfeature.Theyarediscussedhereonlybecausethisisthesectionongetline.SeeTwoway
I/O,wherecoprocessesarediscussedinmoredetail.
Next:GetlineNotes,Previous:Getline/Coprocess,Up:Getline[Contents][Index]
4.9.8UsinggetlineintoaVariablefromaCoprocess
Whenyouusecommand|&getlinevar,theoutputfromthecoprocesscommandissentthroughatwowaypipeto
getlineandintothevariablevar.
Inthisversionofgetline,noneofthepredefinedvariablesarechangedandtherecordisnotsplitintofields.Theonly
variablechangedisvar.However,RTisset.
Next:GetlineSummary,Previous:Getline/Variable/Coprocess,Up:Getline[Contents][Index]
4.9.9PointstoRememberAboutgetline
Herearesomemiscellaneouspointsaboutgetlinethatyoushouldbearinmind:
82/479
29/7/2016
TheGNUAwkUsersGuide
Whengetlinechangesthevalueof$0andNF,awkdoesnotautomaticallyjumptothestartoftheprogramand
starttestingthenewrecordagainsteverypattern.However,thenewrecordistestedagainstanysubsequentrules.
Someveryoldawkimplementationslimitthenumberofpipelinesthatanawkprogrammayhaveopentojustone.
Ingawk,thereisnosuchlimit.Youcanopenasmanypipelines(andcoprocesses)astheunderlyingoperating
systempermits.
AninterestingsideeffectoccursifyouusegetlinewithoutaredirectioninsideaBEGINrule.Becausean
unredirectedgetlinereadsfromthecommandlinedatafiles,thefirstgetlinecommandcausesawktosetthe
valueofFILENAME.Normally,FILENAMEdoesnothaveavalueinsideBEGINrules,becauseyouhavenotyetstarted
toprocessthecommandlinedatafiles.(d.c.)(SeeBEGIN/ENDalsoseeAutoset.)
UsingFILENAMEwithgetline(getline<FILENAME)islikelytobeasourceofconfusion.awkopensaseparate
inputstreamfromthecurrentinputfile.However,bynotusingavariable,$0andNFarestillupdated.Ifyoure
doingthis,itsprobablybyaccident,andyoushouldreconsiderwhatitisyouretryingtoaccomplish.
GetlineSummary,presentsatablesummarizingthegetlinevariantsandwhichvariablestheycanaffect.Itis
worthnotingthatthosevariantsthatdonotuseredirectioncancauseFILENAMEtobeupdatediftheycauseawkto
startreadinganewinputfile.
Ifthevariablebeingassignedisanexpressionwithsideeffects,differentversionsofawkbehavedifferentlyupon
encounteringendoffile.Someversionsdontevaluatetheexpressionmanyversions(includinggawk)do.Hereis
anexample,courtesyofDuncanMoore:
BEGIN{
system("echo1>f")
while((getlinea[++c]<"f")>0){}
printc
}
Here,thesideeffectisthe++c.Iscincrementedifendoffileisencounteredbeforetheelementinaisassigned?
gawktreatsgetlinelikeafunctioncall,andevaluatestheexpressiona[++c]beforeattemptingtoreadfromf.
However,someversionsofawkonlyevaluatetheexpressiononcetheyknowthatthereisastringvaluetobe
assigned.
Previous:GetlineNotes,Up:Getline[Contents][Index]
4.9.10SummaryofgetlineVariants
Table4.1summarizestheeightvariantsofgetline,listingwhichpredefinedvariablesaresetbyeachone,andwhether
thevariantisstandardoragawkextension.Note:foreachvariant,gawksetstheRTpredefinedvariable.
Variant
Effect
Sets$0,NF,FNR,NR,andRT
getlinevar
Setsvar,FNR,NR,andRT
getline<file
Sets$0,NF,andRT
getlinevar<file
SetsvarandRT
command|getline
Sets$0,NF,andRT
command|getlinevar SetsvarandRT
command|&getline
Sets$0,NF,andRT
command|&getlinevar SetsvarandRT
getline
awk/gawk
awk
awk
awk
awk
awk
awk
gawk
gawk
Table4.1:getlinevariantsandwhattheyset
Next:Commandlinedirectories,Previous:Getline,Up:ReadingFiles[Contents][Index]
4.10ReadingInputwithaTimeout
83/479
29/7/2016
TheGNUAwkUsersGuide
Youmayspecifyatimeoutinmillisecondsforreadinginputfromthekeyboard,apipe,ortwowaycommunication,
includingTCP/IPsockets.Thiscanbedoneonaperinput,percommand,orperconnectionbasis,bysettingaspecial
elementinthePROCINFOarray(seeAutoset):
PROCINFO["input_name","READ_TIMEOUT"]=timeoutinmilliseconds
Whenset,thiscausesgawktotimeoutandreturnfailureifnodataisavailabletoreadwithinthespecifiedtimeout
period.Forexample,aTCPclientcandecidetogiveuponreceivinganyresponsefromtheserverafteracertain
amountoftime:
Service="/inet/tcp/0/localhost/daytime"
PROCINFO[Service,"READ_TIMEOUT"]=100
if((Service|&getline)>0)
print$0
elseif(ERRNO!="")
printERRNO
Hereishowtoreadinteractivelyfromtheuser27withoutwaitingformorethanfiveseconds:
PROCINFO["/dev/stdin","READ_TIMEOUT"]=5000
while((getline<"/dev/stdin")>0)
print$0
gawkterminatesthereadoperationifinputdoesnotarriveafterwaitingforthetimeoutperiod,returnsfailure,andsets
ERRNOtoanappropriatestringvalue.Anegativeorzerovalueforthetimeoutisthesameasspecifyingnotimeoutatall.
Atimeoutcanalsobesetforreadingfromthekeyboardintheimplicitloopthatreadsinputrecordsandmatchesthem
againstpatterns,likeso:
$gawk'BEGIN{PROCINFO["","READ_TIMEOUT"]=5000}
>{print"Youentered:"$0}'
gawk
|Youentered:gawk
Inthiscase,failuretorespondwithinfivesecondsresultsinthefollowingerrormessage:
errorgawk:cmd.line:2:(FILENAME=FNR=1)fatal:errorreadinginputfile`':Connectiontimedout
Thetimeoutcanbesetorchangedatanytime,andwilltakeeffectonthenextattempttoreadfromtheinputdevice.In
thefollowingexample,westartwithatimeoutvalueofonesecond,andprogressivelyreduceitbyonetenthofa
seconduntilwewaitindefinitelyfortheinputtoarrive:
while((Service|&getline)>0){
print$0
}
NOTE:Youshouldnotassumethatthereadoperationwillblockexactlyafterthetenthrecordhasbeen
printed.Itispossiblethatgawkwillreadandbuffermorethanonerecordsworthofdatathefirsttime.
Becauseofthis,changingthevalueoftimeoutlikeintheprecedingexampleisnotveryuseful.
IfthePROCINFOelementisnotpresentandtheGAWK_READ_TIMEOUTenvironmentvariableexists,gawkusesitsvalueto
initializethetimeoutvalue.Theexclusiveuseoftheenvironmentvariabletospecifytimeouthasthedisadvantageof
notbeingabletocontrolitonapercommandorperconnectionbasis.
gawkconsidersatimeouteventtobeanerroreventhoughtheattempttoreadfromtheunderlyingdevicemaysucceedin
alaterattempt.Thisisalimitation,anditalsomeansthatyoucannotusethistomultiplexinputfromtwoormore
sources.
84/479
29/7/2016
TheGNUAwkUsersGuide
Assigningatimeoutvaluepreventsreadoperationsfromblockingindefinitely.Butbearinmindthatthereareother
waysgawkcanstallwaitingforaninputdevicetobeready.Anetworkclientcansometimestakealongtimetoestablish
aconnectionbeforeitcanstartreadinganydata,ortheattempttoopenaFIFOspecialfileforreadingcanblock
indefinitelyuntilsomeotherprocessopensitforwriting.
Next:InputSummary,Previous:ReadTimeout,Up:ReadingFiles[Contents][Index]
4.11DirectoriesontheCommandLine
AccordingtothePOSIXstandard,filesnamedontheawkcommandlinemustbetextfilesitisafatalerroriftheyare
not.Mostversionsofawktreatadirectoryonthecommandlineasafatalerror.
Bydefault,gawkproducesawarningforadirectoryonthecommandline,butotherwiseignoresit.Thismakesiteasier
touseshellwildcardswithyourawkprogram:
$gawkfwhizprog.awk*Directoriescouldkillthisprogram
Ifeitheroftheposixortraditionaloptionsisgiven,thengawkrevertstotreatingadirectoryonthecommandline
asafatalerror.
SeeExtensionSampleReaddir,forawaytotreatdirectoriesasusabledatafromanawkprogram.
Next:InputExercises,Previous:Commandlinedirectories,Up:ReadingFiles[Contents][Index]
4.12Summary
InputissplitintorecordsbasedonthevalueofRS.Thepossibilitiesareasfollows:
ValueofRS
Recordsarespliton
awk/gawk
awk
Anysinglecharacter Thatcharacter
Theemptystring("") Runsoftwoormorenewlines awk
Aregexp
Textthatmatchestheregexp gawk
FNRindicateshowmanyrecordshavebeenreadfromthecurrentinputfileNRindicateshowmanyrecordshave
beenreadintotal.
gawksetsRTtothetextmatchedbyRS.
Aftersplittingtheinputintorecords,awkfurthersplitstherecordsintoindividualfields,named$1,$2,andsoon.
$0isthewholerecord,andNFindicateshowmanyfieldsthereare.Thedefaultwaytosplitfieldsisbetween
whitespacecharacters.
Fieldsmaybereferencedusingavariable,asin$NF.Fieldsmayalsobeassignedvalues,whichcausesthevalue
of$0toberecomputedwhenitislaterreferenced.AssigningtoafieldwithanumbergreaterthanNFcreatesthe
fieldandrebuildstherecord,usingOFStoseparatethefields.IncrementingNFdoesthesamething.Decrementing
NFthrowsawayfieldsandrebuildstherecord.
Fieldsplittingismorecomplicatedthanrecordsplitting:
Fieldseparatorvalue
Fieldsaresplit
awk/gawk
FS==""
awk
Onrunsofwhitespace
FS==anysinglecharacter
awk
Onthatcharacter
FS==regexp
awk
Ontextmatchingtheregexp
Suchthateachindividualcharacterisa
FS==""
gawk
separatefield
FIELDWIDTHS==listofcolumns
gawk
Basedoncharacterposition
Onthetextsurroundingtextmatchingthe
FPAT==regexp
gawk
regexp
UsingFS="\n"causestheentirerecordtobeasinglefield(assumingthatnewlinesseparaterecords).
FSmaybesetfromthecommandlineusingtheFoption.Thiscanalsobedoneusingcommandlinevariable
85/479
29/7/2016
TheGNUAwkUsersGuide
assignment.
UsePROCINFO["FS"]toseehowfieldsarebeingsplit.
Usegetlineinitsvariousformstoreadadditionalrecordsfromthedefaultinputstream,fromafile,orfroma
pipeorcoprocess.
UsePROCINFO[file,"READ_TIMEOUT"]tocausereadstotimeoutforfile.
DirectoriesonthecommandlinearefatalforstandardawkgawkignoresthemifnotinPOSIXmode.
Previous:InputSummary,Up:ReadingFiles[Contents][Index]
4.13Exercises
1.UsingtheFIELDWIDTHSvariable(seeConstantSize),writeaprogramtoreadelectiondata,whereeachrecord
representsonevotersvotes.Comeupwithawaytodefinewhichcolumnsareassociatedwitheachballotitem,
andprintthetotalvotes,includingabstentions,foreachitem.
2.PlainGetline,presentedaprogramtoremoveCstylecomments(/**/)fromtheinput.Thatprogramdoes
notworkifonecommentendsononelineandanotheronestartslateronthesameline.Thatcanbefixedby
makingonesimplechange.Whatisit?
Next:Expressions,Previous:ReadingFiles,Up:Top[Contents][Index]
5PrintingOutput
Oneofthemostcommonprogrammingactionsistoprint,oroutput,someoralloftheinput.Usetheprintstatement
forsimpleoutput,andtheprintfstatementforfancierformatting.Theprintstatementisnotlimitedwhencomputing
whichvaluestoprint.However,withtwoexceptions,youcannotspecifyhowtoprintthemhowmanycolumns,
whethertouseexponentialnotationornot,andsoon.(Fortheexceptions,seeOutputSeparators,andOFMT.)For
printingwithspecifications,youneedtheprintfstatement(seePrintf).
Besidesbasicandformattedprinting,thischapteralsocoversI/Oredirectionstofilesandpipes,introducesthespecial
filenamesthatgawkprocessesinternally,anddiscussestheclose()builtinfunction.
Print:
Theprintstatement.
PrintExamples:
Simpleexamplesofprintstatements.
OutputSeparators:
Theoutputseparatorsandhowtochangethem.
OFMT:
ControllingNumericOutputWithprint.
Printf:
Theprintfstatement.
Redirection:
Howtoredirectoutputtomultiplefilesandpipes.
SpecialFD:
SpecialfilesforI/O.
SpecialFiles:
Filenameinterpretationingawk.gawkallowsaccesstoinheritedfiledescriptors.
CloseFilesAndPipes: ClosingInputandOutputFilesandPipes.
OutputSummary:
Outputsummary.
OutputExercises:
Exercises.
Next:PrintExamples,Up:Printing[Contents][Index]
5.1TheprintStatement
Usetheprintstatementtoproduceoutputwithsimple,standardizedformatting.Youspecifyonlythestringsor
numberstoprint,inalistseparatedbycommas.Theyareoutput,separatedbysinglespaces,followedbyanewline.The
statementlookslikethis:
printitem1,item2,
86/479
29/7/2016
TheGNUAwkUsersGuide
Theentirelistofitemsmaybeoptionallyenclosedinparentheses.Theparenthesesarenecessaryifanyoftheitem
expressionsusesthe>relationaloperatorotherwiseitcouldbeconfusedwithanoutputredirection(seeRedirection).
Theitemstoprintcanbeconstantstringsornumbers,fieldsofthecurrentrecord(suchas$1),variables,oranyawk
expression.Numericvaluesareconvertedtostringsandthenprinted.
Thesimplestatementprintwithnoitemsisequivalenttoprint$0:itprintstheentirecurrentrecord.Toprinta
blankline,useprint"".Toprintafixedpieceoftext,useastringconstant,suchas"Don'tPanic",asoneitem.Ifyou
forgettousethedoublequotecharacters,yourtextistakenasanawkexpression,andyouwillprobablygetanerror.
Keepinmindthataspaceisprintedbetweenanytwoitems.
Notethattheprintstatementisastatementandnotanexpressionyoucantuseitinthepatternpartofapattern
actionstatement,forexample.
Next:OutputSeparators,Previous:Print,Up:Printing[Contents][Index]
5.2printStatementExamples
Eachprintstatementmakesatleastonelineofoutput.However,itisntlimitedtoonlyoneline.Ifanitemvalueisa
stringcontaininganewline,thenewlineisoutputalongwiththerestofthestring.Asingle printstatementcanmake
anynumberoflinesthisway.
Thefollowingisanexampleofprintingastringthatcontainsembeddednewlines(the\nisanescapesequence,used
torepresentthenewlinecharacterseeEscapeSequences):
$awk'BEGIN{print"lineone\nlinetwo\nlinethree"}'
|lineone
|linetwo
|linethree
Thenextexample,whichisrunontheinventoryshippedfile,printsthefirsttwofieldsofeachinputrecord,witha
spacebetweenthem:
$awk'{print$1,$2}'inventoryshipped
|Jan13
|Feb15
|Mar15
Acommonmistakeinusingtheprintstatementistoomitthecommabetweentwoitems.Thisoftenhastheeffectof
makingtheitemsruntogetherintheoutput,withnospace.Thereasonforthisisthatjuxtaposingtwostringexpressions
inawkmeanstoconcatenatethem.Hereisthesameprogram,withoutthecomma:
$awk'{print$1$2}'inventoryshipped
|Jan13
|Feb15
|Mar15
Tosomeoneunfamiliarwiththeinventoryshippedfile,neitherexamplesoutputmakesmuchsense.Aheadinglineat
thebeginningwouldmakeitclearer.Letsaddsomeheadingstoourtableofmonths($1)andgreencratesshipped($2).
WedothisusingaBEGINrule(seeBEGIN/END)sothattheheadingsareonlyprintedonce:
awk'BEGIN{print"MonthCrates"
print""}
{print$1,$2}'inventoryshipped
Whenrun,theprogramprintsthefollowing:
MonthCrates
87/479
29/7/2016
TheGNUAwkUsersGuide
Jan13
Feb15
Mar15
Theonlyproblem,however,isthattheheadingsandthetabledatadontlineup!Wecanfixthisbyprintingsome
spacesbetweenthetwofields:
print""}
{print$1,"",$2}'inventoryshipped
Liningupcolumnsthiswaycangetprettycomplicatedwhentherearemanycolumnstofix.Countingspacesfortwoor
threecolumnsissimple,butanymorethanthiscantakeupalotoftime.Thisiswhytheprintfstatementwascreated
(seePrintf)oneofitsspecialtiesisliningupcolumnsofdata.
NOTE:Youcancontinueeitheraprintorprintfstatementsimplybyputtinganewlineafteranycomma
(seeStatements/Lines).
Next:OFMT,Previous:PrintExamples,Up:Printing[Contents][Index]
5.3OutputSeparators
Asmentionedpreviously,aprintstatementcontainsalistofitemsseparatedbycommas.Intheoutput,theitemsare
normallyseparatedbysinglespaces.However,thisdoesntneedtobethecaseasinglespaceissimplythedefault.Any
stringofcharactersmaybeusedastheoutputfieldseparatorbysettingthepredefinedvariableOFS.Theinitialvalueof
thisvariableisthestring""(i.e.,asinglespace).
Theoutputfromanentireprintstatementiscalledanoutputrecord.Eachprintstatementoutputsoneoutputrecord,
andthenoutputsastringcalledtheoutputrecordseparator(orORS).TheinitialvalueofORSisthestring"\n"(i.e.,a
newlinecharacter).Thus,eachprintstatementnormallymakesaseparateline.
Inordertochangehowoutputfieldsandrecordsareseparated,assignnewvaluestothevariablesOFSandORS.The
usualplacetodothisisintheBEGINrule(seeBEGIN/END),sothatithappensbeforeanyinputisprocessed.Itcanalso
bedonewithassignmentsonthecommandline,beforethenamesoftheinputfiles,orusingthevcommandline
option(seeOptions).Thefollowingexampleprintsthefirstandsecondfieldsofeachinputrecord,separatedbya
semicolon,withablanklineaddedaftereachnewline:
$awk'BEGIN{OFS=";";ORS="\n\n"}
>{print$1,$2}'maillist
|Amelia;5555553
|
|Anthony;5553412
|
|Becky;5557685
|
|Bill;5551675
|
|Broderick;5550542
|
|Camilla;5552912
|
|Fabius;5551234
|
|Julie;5556699
|
|Martin;5556480
|
|Samuel;5553430
|
|JeanPaul;5552127
|
88/479
29/7/2016
TheGNUAwkUsersGuide
IfthevalueofORSdoesnotcontainanewline,theprogramsoutputrunstogetheronasingleline.
Next:Printf,Previous:OutputSeparators,Up:Printing[Contents][Index]
5.4ControllingNumericOutputwithprint
Whenprintingnumericvalueswiththeprintstatement,awkinternallyconvertseachnumbertoastringofcharacters
andprintsthatstring.awkusesthesprintf()functiontodothisconversion(seeStringFunctions).Fornow,itsuffices
tosaythatthesprintf()functionacceptsaformatspecificationthattellsithowtoformatnumbers(orstrings),andthat
thereareanumberofdifferentwaysinwhichnumberscanbeformatted.Thedifferentformatspecificationsare
discussedmorefullyinControlLetters.
ThepredefinedvariableOFMTcontainstheformatspecificationthatprintuseswithsprintf()whenitwantstoconverta
numbertoastringforprinting.ThedefaultvalueofOFMTis"%.6g".Thewayprintprintsnumberscanbechangedby
supplyingadifferentformatspecificationforthevalueofOFMT,asshowninthefollowingexample:
$awk'BEGIN{
>OFMT="%.0f"#printnumbersasintegers(rounds)
>print17.23,17.54}'
|1718
AccordingtothePOSIXstandard,awksbehaviorisundefinedifOFMTcontainsanythingbutafloatingpointconversion
specification.(d.c.)
Next:Redirection,Previous:OFMT,Up:Printing[Contents][Index]
5.5UsingprintfStatementsforFancierPrinting
Formoreprecisecontrolovertheoutputformatthanwhatisprovidedbyprint,useprintf.Withprintfyoucan
specifythewidthtouseforeachitem,aswellasvariousformattingchoicesfornumbers(suchaswhatoutputbaseto
use,whethertoprintanexponent,whethertoprintasign,andhowmanydigitstoprintafterthedecimalpoint).
BasicPrintf:
Syntaxoftheprintfstatement.
ControlLetters: Formatcontrolletters.
FormatModifiers: Formatspecificationmodifiers.
PrintfExamples: Severalexamples.
Next:ControlLetters,Up:Printf[Contents][Index]
5.5.1IntroductiontotheprintfStatement
Asimpleprintfstatementlookslikethis:
printfformat,item1,item2,
Asforprint,theentirelistofargumentsmayoptionallybeenclosedinparentheses.Heretoo,theparenthesesare
necessaryifanyoftheitemexpressionsusesthe>relationaloperatorotherwise,itcanbeconfusedwithanoutput
redirection(seeRedirection).
Thedifferencebetweenprintfandprintistheformatargument.Thisisanexpressionwhosevalueistakenasastring
itspecifieshowtooutputeachoftheotherarguments.Itiscalledtheformatstring.
TheformatstringisverysimilartothatintheISOClibraryfunctionprintf().Mostofformatistexttooutput
verbatim.Scatteredamongthistextareformatspecifiersoneperitem.Eachformatspecifiersaystooutputthenext
itemintheargumentlistatthatplaceintheformat.
89/479
29/7/2016
TheGNUAwkUsersGuide
Theprintfstatementdoesnotautomaticallyappendanewlinetoitsoutput.Itoutputsonlywhattheformatstring
specifies.Soifanewlineisneeded,youmustincludeoneintheformatstring.TheoutputseparatorvariablesOFSand
ORShavenoeffectonprintfstatements.Forexample:
$awk'BEGIN{
>ORS="\nOUCH!\n";OFS="+"
>msg="Don\47tPanic!"
>printf"%s\n",msg
>}'
|Don'tPanic!
Here,neitherthe+northeOUCH!appearsintheoutputmessage.
Next:FormatModifiers,Previous:BasicPrintf,Up:Printf[Contents][Index]
5.5.2FormatControlLetters
Aformatspecifierstartswiththecharacter%andendswithaformatcontrolletterittellstheprintfstatementhow
tooutputoneitem.Theformatcontrolletterspecifieswhatkindofvaluetoprint.Therestoftheformatspecifieris
madeupofoptionalmodifiersthatcontrolhowtoprintthevalue,suchasthefieldwidth.Hereisalistoftheformat
controlletters:
%c
Printanumberasacharacterthus,printf"%c",65outputstheletterA.Theoutputforastringvalueisthe
firstcharacterofthestring.
NOTE:ThePOSIXstandardsaysthefirstcharacterofastringisprinted.Inlocaleswithmultibyte
characters,gawkattemptstoconverttheleadingbytesofthestringintoavalidwidecharacterand
thentoprintthemultibyteencodingofthatcharacter.Similarly,whenprintinganumericvalue,gawk
allowsthevaluetobewithinthenumericrangeofvaluesthatcanbeheldinawidecharacter.Ifthe
conversiontomultibyteencodingfails,gawkusestheloweightbitsofthevalueasthecharacterto
print.
Otherawkversionsgenerallyrestrictthemselvestoprintingthefirstbyteofastringortonumeric
valueswithintherangeofasinglebyte(0255).
%d,%i
Printadecimalinteger.Thetwocontrollettersareequivalent.(The%ispecificationisforcompatibilitywith
ISOC.)
%e,%E
Printanumberinscientific(exponential)notation.Forexample:
printf"%4.3e\n",1950
prints1.950e+03,withatotaloffoursignificantfigures,threeofwhichfollowthedecimalpoint.(The4.3
representstwomodifiers,discussedinthenextsubsection.)%EusesEinsteadofeintheoutput.
%f
Printanumberinfloatingpointnotation.Forexample:
printf"%4.3f",1950
prints1950.000,withatotaloffoursignificantfigures,threeofwhichfollowthedecimalpoint.(The4.3
representstwomodifiers,discussedinthenextsubsection.)
90/479
29/7/2016
TheGNUAwkUsersGuide
OnsystemssupportingIEEE754floatingpointformat,valuesrepresentingnegativeinfinityareformattedas
inforinfinity,andpositiveinfinityasinforinfinity.Thespecialnotanumbervalueformatsas
nanornan(seeMathDefinitions).
%F
Like%f,buttheinfinityandnotanumbervaluesarespelledusinguppercaseletters.
The%FformatisaPOSIXextensiontoISOCnotallsystemssupportit.Onthosethatdont,gawkuses%f
instead.
%g,%G
Printanumberineitherscientificnotationorinfloatingpointnotation,whicheverusesfewercharactersifthe
resultisprintedinscientificnotation,%GusesEinsteadofe.
%o
Printanunsignedoctalinteger(seeNondecimalnumbers).
%s
Printastring.
%u
Printanunsigneddecimalinteger.(Thisformatisofmarginaluse,becauseallnumbersinawkarefloatingpointit
isprovidedprimarilyforcompatibilitywithC.)
%x,%X
Printanunsignedhexadecimalinteger%XusesthelettersAthroughFinsteadof athroughf(see
Nondecimalnumbers).
%%
Printasingle%.Thisdoesnotconsumeanargumentanditignoresanymodifiers.
NOTE:WhenusingtheintegerformatcontrollettersforvaluesthatareoutsidetherangeofthewidestC
integertype,gawkswitchestothe%gformatspecifier.Iflintisprovidedonthecommandline(see
Options),gawkwarnsaboutthis.Otherversionsofawkmayprintinvalidvaluesordosomethingelse
entirely.(d.c.)
Next:PrintfExamples,Previous:ControlLetters,Up:Printf[Contents][Index]
5.5.3ModifiersforprintfFormats
Aformatspecificationcanalsoincludemodifiersthatcancontrolhowmuchoftheitemsvalueisprinted,aswellas
howmuchspaceitgets.Themodifierscomebetweenthe%andtheformatcontrolletter.Weusethebulletsymbol
inthefollowingexamplestorepresentspacesintheoutput.Herearethepossiblemodifiers,intheorderinwhichthey
mayappear:
N$
Anintegerconstantfollowedbya$isapositionalspecifier.Normally,formatspecificationsareappliedto
argumentsintheordergivenintheformatstring.Withapositionalspecifier,theformatspecificationisappliedto
aspecificargument,insteadofwhatwouldbethenextargumentinthelist.Positionalspecifiersbegincounting
withone.Thus:
91/479
29/7/2016
TheGNUAwkUsersGuide
printf"%s%s\n","don't","panic"
printf"%2$s%1$s\n","panic","don't"
printsthefamousfriendlymessagetwice.
Atfirstglance,thisfeaturedoesntseemtobeofmuchuse.Itisinfactagawkextension,intendedforusein
translatingmessagesatruntime.SeePrintfOrdering,whichdescribeshowandwhytousepositionalspecifiers.
Fornow,weignorethem.
(Minus)
Theminussign,usedbeforethewidthmodifier(seelateroninthislist),saystoleftjustifytheargumentwithin
itsspecifiedwidth.Normally,theargumentisprintedrightjustifiedinthespecifiedwidth.Thus:
printf"%4s","foo"
printsfoo.
space
Fornumericconversions,prefixpositivevalueswithaspaceandnegativevalueswithaminussign.
+
Theplussign,usedbeforethewidthmodifier(seelateroninthislist),saystoalwayssupplyasignfornumeric
conversions,evenifthedatatoformatispositive.The+overridesthespacemodifier.
#
Useanalternativeformforcertaincontrolletters.For%o,supplyaleadingzero.For%xand%X,supplya
leading0xor0Xforanonzeroresult.For%e,%E,%f,and%F,theresultalwayscontainsadecimalpoint.
For%gand%G,trailingzerosarenotremovedfromtheresult.
0
Aleading0(zero)actsasaflagindicatingthatoutputshouldbepaddedwithzerosinsteadofspaces.This
appliesonlytothenumericoutputformats.Thisflagonlyhasaneffectwhenthefieldwidthiswiderthanthe
valuetoprint.
'
AsinglequoteorapostrophecharacterisaPOSIXextensiontoISOC.Itindicatesthattheintegerpartofa
floatingpointvalue,ortheentirepartofanintegerdecimalvalue,shouldhaveathousandsseparatorcharacterin
it.Thisonlyworksinlocalesthatsupportsuchcharacters.Forexample:
$catthousands.awkShowsourceprogram
|BEGIN{printf"%'d\n",1234567}
$LC_ALL=Cgawkfthousands.awk
|1234567Resultsin"C"locale
$LC_ALL=en_US.UTF8gawkfthousands.awk
|1,234,567ResultsinUSEnglishUTFlocale
Formoreinformationaboutlocalesandinternationalizationissues,seeLocales.
NOTE:The'flagisanicefeature,butitsusecomplicatesthings:itbecomesdifficulttouseitin
commandlineprograms.Forinformationonappropriatequotingtricks,seeQuoting.
width
Thisisanumberspecifyingthedesiredminimumwidthofafield.Insertinganynumberbetweenthe%signand
theformatcontrolcharacterforcesthefieldtoexpandtothiswidth.Thedefaultwaytodothisistopadwith
spacesontheleft.Forexample:
92/479
29/7/2016
TheGNUAwkUsersGuide
printf"%4s","foo"
printsfoo.
Thevalueofwidthisaminimumwidth,notamaximum.Iftheitemvaluerequiresmorethanwidthcharacters,it
canbeaswideasnecessary.Thus,thefollowing:
printf"%4s","foobar"
printsfoobar.
Precedingthewidthwithaminussigncausestheoutputtobepaddedwithspacesontheright,insteadofonthe
left.
.prec
Aperiodfollowedbyanintegerconstantspecifiestheprecisiontousewhenprinting.Themeaningofthe
precisionvariesbycontrolletter:
%d,%i,%o,%u,%x,%X
Minimumnumberofdigitstoprint.
%e,%E,%f,%F
Numberofdigitstotherightofthedecimalpoint.
%g,%G
Maximumnumberofsignificantdigits.
%s
Maximumnumberofcharactersfromthestringthatshouldprint.
Thus,thefollowing:
printf"%.4s","foobar"
printsfoob.
TheClibraryprintfsdynamicwidthandpreccapability(e.g.,"%*.*s")issupported.Insteadofsupplyingexplicit
widthand/orprecvaluesintheformatstring,theyarepassedintheargumentlist.Forexample:
w=5
p=3
s="abcdefg"
printf"%*.*s\n",w,p,s
isexactlyequivalentto:
s="abcdefg"
printf"%5.3s\n",s
Bothprogramsoutputabc.Earlierversionsofawkdidnotsupportthiscapability.Ifyoumustusesuchaversion,you
maysimulatethisfeaturebyusingconcatenationtobuilduptheformatstring,likeso:
w=5
p=3
s="abcdefg"
printf"%"w"."p"s\n",s
Thisisnotparticularlyeasytoread,butitdoeswork.
93/479
29/7/2016
TheGNUAwkUsersGuide
Cprogrammersmaybeusedtosupplyingadditionalmodifiers(h,j,l,L,t,andz)inprintfformatstrings.
Thesearenotvalidinawk.Mostawkimplementationssilentlyignorethem.Iflintisprovidedonthecommandline
(seeOptions),gawkwarnsabouttheiruse.Ifposixissupplied,theiruseisafatalerror.
Previous:FormatModifiers,Up:Printf[Contents][Index]
5.5.4ExamplesUsingprintf
Thefollowingsimpleexampleshowshowtouseprintftomakeanalignedtable:
awk'{printf"%10s%s\n",$1,$2}'maillist
Thiscommandprintsthenamesofthepeople($1)inthefilemaillistasastringof10charactersthatareleftjustified.
Italsoprintsthephonenumbers($2)nextontheline.Thisproducesanalignedtwocolumntableofnamesandphone
numbers,asshownhere:
$awk'{printf"%10s%s\n",$1,$2}'maillist
|Amelia5555553
|Anthony5553412
|Becky5557685
|Bill5551675
|Broderick5550542
|Camilla5552912
|Fabius5551234
|Julie5556699
|Martin5556480
|Samuel5553430
|JeanPaul5552127
Inthiscase,thephonenumbershadtobeprintedasstringsbecausethenumbersareseparatedbydashes.Printingthe
phonenumbersasnumberswouldhaveproducedjustthefirstthreedigits:555.Thiswouldhavebeenpretty
confusing.
Itwasntnecessarytospecifyawidthforthephonenumbersbecausetheyarelastontheirlines.Theydontneedto
havespacesafterthem.
Thetablecouldbemadetolookevennicerbyaddingheadingstothetopsofthecolumns.ThisisdoneusingaBEGIN
rule(seeBEGIN/END)sothattheheadersareonlyprintedonce,atthebeginningoftheawkprogram:
awk'BEGIN{print"NameNumber"
print""}
{printf"%10s%s\n",$1,$2}'maillist
Theprecedingexamplemixesprintandprintfstatementsinthesameprogram.Usingjustprintfstatementscan
producethesameresults:
awk'BEGIN{printf"%10s%s\n","Name","Number"
printf"%10s%s\n","",""}
{printf"%10s%s\n",$1,$2}'maillist
Printingeachcolumnheadingwiththesameformatspecificationusedforthecolumnelementsensuresthatthe
headingsarealignedjustlikethecolumns.
Thefactthatthesameformatspecificationisusedthreetimescanbeemphasizedbystoringitinavariable,likethis:
awk'BEGIN{format="%10s%s\n"
printfformat,"Name","Number"
printfformat,"",""}
{printfformat,$1,$2}'maillist
Next:SpecialFD,Previous:Printf,Up:Printing[Contents][Index]
94/479
29/7/2016
TheGNUAwkUsersGuide
5.6RedirectingOutputofprintandprintf
Sofar,theoutputfromprintandprintfhasgonetothestandardoutput,usuallythescreen.Bothprintandprintfcan
alsosendtheiroutputtootherplaces.Thisiscalledredirection.
NOTE:Whensandboxisspecified(seeOptions),redirectingoutputtofiles,pipes,andcoprocessesis
disabled.
Aredirectionappearsaftertheprintorprintfstatement.Redirectionsinawkarewrittenjustlikeredirectionsinshell
commands,exceptthattheyarewritteninsidetheawkprogram.
Therearefourformsofoutputredirection:outputtoafile,outputappendedtoafile,outputthroughapipetoanother
command,andoutputtoacoprocess.Weshowthemallfortheprintstatement,buttheyworkidenticallyforprintf:
printitems>outputfile
Thisredirectionprintstheitemsintotheoutputfilenamedoutputfile.Thefilenameoutputfilecanbeany
expression.Itsvalueischangedtoastringandthenusedasafilename(seeExpressions).
Whenthistypeofredirectionisused,theoutputfileiserasedbeforethefirstoutputiswrittentoit.Subsequent
writestothesameoutputfiledonoteraseoutputfile,butappendtoit.(Thisisdifferentfromhowyouuse
redirectionsinshellscripts.)Ifoutputfiledoesnotexist,itiscreated.Forexample,hereishowanawkprogram
canwritealistofpeoplesnamestoonefilenamednamelist,andalistofphonenumberstoanotherfilenamed
phonelist:
$awk'{print$2>"phonelist"
>print$1>"namelist"}'maillist
$catphonelist
|5555553
|5553412
$catnamelist
|Amelia
|Anthony
Eachoutputfilecontainsonenameornumberperline.
printitems>>outputfile
Thisredirectionprintstheitemsintothepreexistingoutputfilenamedoutputfile.Thedifferencebetweenthis
andthesingle>redirectionisthattheoldcontents(ifany)ofoutputfilearenoterased.Instead,theawkoutput
isappendedtothefile.Ifoutputfiledoesnotexist,thenitiscreated.
printitems|command
Itispossibletosendoutputtoanotherprogramthroughapipeinsteadofintoafile.Thisredirectionopensapipe
tocommand,andwritesthevaluesofitemsthroughthispipetoanotherprocesscreatedtoexecutecommand.
Theredirectionargumentcommandisactuallyanawkexpression.Itsvalueisconvertedtoastringwhosecontents
givetheshellcommandtoberun.Forexample,thefollowingproducestwofiles,oneunsortedlistofpeoples
names,andonelistsortedinreversealphabeticalorder:
awk'{print$1>"names.unsorted"
command="sortr>names.sorted"
print$1|command}'maillist
Theunsortedlistiswrittenwithanordinaryredirection,whilethesortedlistiswrittenbypipingthroughthesort
utility.
Thenextexampleusesredirectiontomailamessagetothemailinglistbugsystem.Thismightbeusefulwhen
troubleisencounteredinanawkscriptrunperiodicallyforsystemmaintenance:
95/479
29/7/2016
TheGNUAwkUsersGuide
report="mailbugsystem"
print("Awkscriptfailed:",$0)|report
print("atrecordnumber",FNR,"of",FILENAME)|report
close(report)
Theclose()functioniscalledherebecauseitsagoodideatoclosethepipeassoonasalltheintendedoutputhas
beensenttoit.SeeCloseFilesAndPipes,formoreinformation.
Thisexamplealsoillustratestheuseofavariabletorepresentafileorcommanditisnotnecessarytoalways
useastringconstant.Usingavariableisgenerallyagoodidea,because(ifyoumeantorefertothatsamefileor
command)awkrequiresthatthestringvaluebewrittenidenticallyeverytime.
printitems|&command
Thisredirectionprintstheitemstotheinputofcommand.Thedifferencebetweenthisandthesingle|
redirectionisthattheoutputfromcommandcanbereadwithgetline.Thus,commandisacoprocess,which
workstogetherwithbutissubsidiarytotheawkprogram.
Thisfeatureisagawkextension,andisnotavailableinPOSIXawk.SeeGetline/Coprocess,forabriefdiscussion.
SeeTwowayI/O,foramorecompletediscussion.
Redirectingoutputusing>,>>,|,or|&asksthesystemtoopenafile,pipe,orcoprocessonlyiftheparticularfile
orcommandyouspecifyhasnotalreadybeenwrittentobyyourprogramorifithasbeenclosedsinceitwaslast
writtento.
Itisacommonerrortouse>redirectionforthefirstprinttoafile,andthentouse>>forsubsequentoutput:
#clearthefile
print"Don'tpanic">"guide.txt"
#append
print"Avoidimprobabilitygenerators">>"guide.txt"
Thisisindeedhowredirectionsmustbeusedfromtheshell.Butinawk,itisntnecessary.Inthiskindofcase,aprogram
shoulduse>foralltheprintstatements,becausetheoutputfileisonlyopenedonce.(Ithappensthatifyoumix>
and>>outputisproducedintheexpectedorder.However,mixingtheoperatorsforthesamefileisdefinitelypoor
style,andisconfusingtoreadersofyourprogram.)
Asmentionedearlier(seeGetlineNotes),manyManyolderawkimplementationslimitthenumberofpipelinesthatan
awkprogrammayhaveopentojustone!Ingawk,thereisnosuchlimit.gawkallowsaprogramtoopenasmanypipelines
astheunderlyingoperatingsystempermits.
Pipingintosh
Aparticularlypowerfulwaytouseredirectionistobuildcommandlinesandpipethemintotheshell,sh.For
example,supposeyouhavealistoffilesbroughtoverfromasystemwhereallthefilenamesarestoredinuppercase,
andyouwishtorenamethemtohavenamesinalllowercase.Thefollowingprogramisbothsimpleandefficient:
{printf("mv%s%s\n",$0,tolower($0))|"sh"}
END{close("sh")}
Thetolower()functionreturnsitsargumentstringwithalluppercasecharactersconvertedtolowercase(seeString
Functions).Theprogrambuildsupalistofcommandlines,usingthemvutilitytorenamethefiles.Itthensendsthelist
totheshellforexecution.
SeeShellQuoting,forafunctionthatcanhelpingeneratingcommandlinestobefedtotheshell.
Next:SpecialFiles,Previous:Redirection,Up:Printing[Contents][Index]
96/479
29/7/2016
TheGNUAwkUsersGuide
5.7SpecialFilesforStandardPreopenedDataStreams
Runningprogramsconventionallyhavethreeinputandoutputstreamsalreadyavailabletothemforreadingandwriting.
Theseareknownasthestandardinput,standardoutput,andstandarderroroutput.Theseopenstreams(andanyother
openfilesorpipes)areoftenreferredtobythetechnicaltermfiledescriptors.
Thesestreamsare,bydefault,connectedtoyourkeyboardandscreen,buttheyareoftenredirectedwiththeshell,via
the<,<<,>,>>,>&,and|operators.Standarderroristypicallyusedforwritingerrormessagesthereason
therearetwoseparatestreams,standardoutputandstandarderror,issothattheycanberedirectedseparately.
Intraditionalimplementationsofawk,theonlywaytowriteanerrormessagetostandarderrorinanawkprogramisas
follows:
print"Seriouserrordetected!"|"cat1>&2"
Thisworksbyopeningapipelinetoashellcommandthatcanaccessthestandarderrorstreamthatitinheritsfromthe
awkprocess.Thisisfarfromelegant,anditalsorequiresaseparateprocess.Sopeoplewritingawkprogramsoftendont
dothis.Instead,theysendtheerrormessagestothescreen,likethis:
print"Seriouserrordetected!">"/dev/tty"
(/dev/ttyisaspecialfilesuppliedbytheoperatingsystemthatisconnectedtoyourkeyboardandscreen.Itrepresents
theterminal,28whichonmodernsystemsisakeyboardandscreen,notaserialconsole.)Thisgenerallyhasthesame
effect,butnotalways:althoughthestandarderrorstreamisusuallythescreen,itcanberedirectedwhenthathappens,
writingtothescreenisnotcorrect.Infact,ifawkisrunfromabackgroundjob,itmaynothaveaterminalatall.Then
opening/dev/ttyfails.
gawk,BWKawk,andmawkprovidespecialfilenamesforaccessingthethreestandardstreams.Ifthefilenamematches
oneofthesespecialnameswhengawk(oroneoftheothers)redirectsinputoroutput,thenitdirectlyusesthedescriptor
thatthefilenamestandsfor.Thesespecialfilenamesworkforalloperatingsystemsthatgawkhasbeenportedto,not
justthosethatarePOSIXcompliant:
/dev/stdin
Thestandardinput(filedescriptor0).
/dev/stdout
Thestandardoutput(filedescriptor1).
/dev/stderr
Thestandarderroroutput(filedescriptor2).
Withthesefacilities,theproperwaytowriteanerrormessagethenbecomes:
print"Seriouserrordetected!">"/dev/stderr"
Notetheuseofquotesaroundthefilename.Likewithanyotherredirection,thevaluemustbeastring.Itisacommon
errortoomitthequotes,whichleadstoconfusingresults.
gawkdoesnottreatthesefilenamesasspecialwheninPOSIXcompatibilitymode.However,becauseBWKawk
supportsthem,gawkdoessupportthemevenwheninvokedwiththetraditionaloption(seeOptions).
Next:CloseFilesAndPipes,Previous:SpecialFD,Up:Printing[Contents][Index]
5.8SpecialFilenamesingawk
Besidesaccesstostandardinput,standardoutput,andstandarderror,gawkprovidesaccesstoanyopenfiledescriptor.
97/479
29/7/2016
TheGNUAwkUsersGuide
Additionally,therearespecialfilenamesreservedforTCP/IPnetworking.
OtherInheritedFiles: Accessingotheropenfileswithgawk.
SpecialNetwork:
Specialfilesfornetworkcommunications.
SpecialCaveats:
Next:SpecialNetwork,Up:SpecialFiles[Contents][Index]
5.8.1AccessingOtherOpenFileswithgawk
Besidesthe/dev/stdin,/dev/stdout,and/dev/stderrspecialfilenamesmentionedearlier,gawkprovidessyntaxfor
accessinganyotherinheritedopenfile:
/dev/fd/N
ThefileassociatedwithfiledescriptorN.Suchafilemustbeopenedbytheprograminitiatingtheawkexecution
(typicallytheshell).Unlessspecialpainsaretakenintheshellfromwhichgawkisinvoked,onlydescriptors0,1,
and2areavailable.
Thefilenames/dev/stdin,/dev/stdout,and/dev/stderrareessentiallyaliasesfor/dev/fd/0,/dev/fd/1,and
/dev/fd/2,respectively.However,thosenamesaremoreselfexplanatory.
Notethatusingclose()onafilenameoftheform"/dev/fd/N",forfiledescriptornumbersabovetwo,doesactually
closethegivenfiledescriptor.
Next:SpecialCaveats,Previous:OtherInheritedFiles,Up:SpecialFiles[Contents][Index]
5.8.2SpecialFilesforNetworkCommunications
gawkprogramscanopenatwowayTCP/IPconnection,actingaseitheraclientoraserver.Thisisdoneusingaspecial
filenameoftheform:
/nettype/protocol/localport/remotehost/remoteport
Thenettypeisoneofinet,inet4,orinet6.Theprotocolisoneoftcporudp,andtheotherfieldsrepresentthe
otheressentialpiecesofinformationformakinganetworkingconnection.Thesefilenamesareusedwiththe|&
operatorforcommunicatingwithacoprocess(seeTwowayI/O).Thisisanadvancedfeature,mentionedhereonlyfor
completeness.FulldiscussionisdelayeduntilTCP/IPNetworking.
Previous:SpecialNetwork,Up:SpecialFiles[Contents][Index]
5.8.3SpecialFilenameCaveats
Herearesomethingstobearinmindwhenusingthespecialfilenamesthatgawkprovides:
RecognitionofthefilenamesforthethreestandardpreopenedfilesisdisabledonlyinPOSIXmode.
Recognitionoftheotherspecialfilenamesisdisabledifgawkisincompatibilitymode(eithertraditionalor
posixseeOptions).
gawkalwaysinterpretsthesespecialfilenames.Forexample,using/dev/fd/4foroutputactuallywritesonfile
descriptor4,andnotonanewfiledescriptorthatisdup()edfromfiledescriptor4.Mostofthetimethisdoesnot
matterhowever,itisimportanttonotcloseanyofthefilesrelatedtofiledescriptors0,1,and2.Doingsoresults
inunpredictablebehavior.
Next:OutputSummary,Previous:SpecialFiles,Up:Printing[Contents][Index]
98/479
29/7/2016
TheGNUAwkUsersGuide
5.9ClosingInputandOutputRedirections
Ifthesamefilenameorthesameshellcommandisusedwithgetlinemorethanonceduringtheexecutionofanawk
program(seeGetline),thefileisopened(orthecommandisexecuted)thefirsttimeonly.Atthattime,thefirstrecord
ofinputisreadfromthatfileorcommand.Thenexttimethesamefileorcommandisusedwithgetline,anotherrecord
isreadfromit,andsoon.
Similarly,whenafileorpipeisopenedforoutput,awkremembersthefilenameorcommandassociatedwithit,and
subsequentwritestothesamefileorcommandareappendedtothepreviouswrites.Thefileorpipestaysopenuntilawk
exits.
Thisimpliesthatspecialstepsarenecessaryinordertoreadthesamefileagainfromthebeginning,ortorerunashell
command(ratherthanreadingmoreoutputfromthesamecommand).Theclose()functionmakesthesethings
possible:
close(filename)
or:
close(command)
Theargumentfilenameorcommandcanbeanyexpression.Itsvaluemustexactlymatchthestringthatwasusedto
openthefileorstartthecommand(spacesandotherirrelevantcharactersincluded).Forexample,ifyouopenapipe
withthis:
"sortrnames"|getlinefoo
thenyoumustcloseitwiththis:
close("sortrnames")
Oncethisfunctioncallisexecuted,thenextgetlinefromthatfileorcommand,orthenextprintorprintftothatfile
orcommand,reopensthefileorrerunsthecommand.Becausetheexpressionthatyouusetocloseafileorpipeline
mustexactlymatchtheexpressionusedtoopenthefileorrunthecommand,itisgoodpracticetouseavariabletostore
thefilenameorcommand.Thepreviousexamplebecomesthefollowing:
sortcom="sortrnames"
sortcom|getlinefoo
close(sortcom)
Thishelpsavoidhardtofindtypographicalerrorsinyourawkprograms.Herearesomeofthereasonsforclosingan
outputfile:
Towriteafileandreaditbacklateroninthesameawkprogram.Closethefileafterwritingit,thenbeginreading
itwithgetline.
Towritenumerousfiles,successively,inthesameawkprogram.Ifthefilesarentclosed,eventuallyawkmay
exceedasystemlimitonthenumberofopenfilesinoneprocess.Itisbesttocloseeachonewhentheprogram
hasfinishedwritingit.
Tomakeacommandfinish.Whenoutputisredirectedthroughapipe,thecommandreadingthepipenormally
continuestotrytoreadinputaslongasthepipeisopen.Oftenthismeansthecommandcannotreallydoitswork
untilthepipeisclosed.Forexample,ifoutputisredirectedtothemailprogram,themessageisnotactuallysent
untilthepipeisclosed.
Torunthesameprogramasecondtime,withthesamearguments.Thisisnotthesamethingasgivingmoreinput
tothefirstrun!
Forexample,supposeaprogrampipesoutputtothemailprogram.Ifitoutputsseverallinesredirectedtothispipe
withoutclosingit,theymakeasinglemessageofseverallines.Bycontrast,iftheprogramclosesthepipeafter
eachlineofoutput,theneachlinemakesaseparatemessage.
99/479
29/7/2016
TheGNUAwkUsersGuide
Ifyouusemorefilesthanthesystemallowsyoutohaveopen,gawkattemptstomultiplextheavailableopenfilesamong
yourdatafiles.gawksabilitytodothisdependsuponthefacilitiesofyouroperatingsystem,soitmaynotalwayswork.
Itisthereforebothgoodpracticeandgoodportabilityadvicetoalwaysuseclose()onyourfileswhenyouaredone
withthem.Infact,ifyouareusingalotofpipes,itisessentialthatyouclosecommandswhendone.Forexample,
considersomethinglikethis:
{
command=("grep"$1"/some/file|my_progq"$3)
while((command|getline)>0){
processoutputofcommand
}
#needclose(command)here
}
Thisexamplecreatesanewpipelinebasedondataineachrecord.Withoutthecalltoclose()indicatedinthecomment,
awkcreateschildprocessestorunthecommands,untiliteventuallyrunsoutoffiledescriptorsformorepipelines.
Eventhougheachcommandhasfinished(asindicatedbytheendoffilereturnstatusfromgetline),thechildprocessis
notterminated29moreimportantly,thefiledescriptorforthepipeisnotclosedandreleaseduntilclose()iscalledor
awkexits.
close()silentlydoesnothingifgivenanargumentthatdoesnotrepresentafile,pipe,orcoprocessthatwasopened
witharedirection.Insuchacase,itreturnsanegativevalue,indicatinganerror.Inaddition,gawksetsERRNOtoastring
indicatingtheerror.
Notealsothatclose(FILENAME)hasnomagiceffectsontheimplicitloopthatreadsthroughthefilesnamedonthe
commandline.Itis,morelikely,acloseofafilethatwasneveropenedwitharedirection,soawksilentlydoesnothing,
exceptreturnanegativevalue.
Whenusingthe|&operatortocommunicatewithacoprocess,itisoccasionallyusefultobeabletocloseoneendof
thetwowaypipewithoutclosingtheother.Thisisdonebysupplyingasecondargumenttoclose().Asinanyother
calltoclose(),thefirstargumentisthenameofthecommandorspecialfileusedtostartthecoprocess.Thesecond
argumentshouldbeastring,witheitherofthevalues"to"or"from".Casedoesnotmatter.Asthisisanadvanced
feature,discussionisdelayeduntilTwowayI/O,whichdescribesitinmoredetailandgivesanexample.
Usingclose()sReturnValue
InmanyolderversionsofUnixawk,theclose()functionisactuallyastatement.(d.c.)Itisasyntaxerrortotryand
usethereturnvaluefromclose():
command=""
command|getlineinfo
retval=close(command)#syntaxerrorinmanyUnixawks
gawktreatsclose()asafunction.Thereturnvalueis1iftheargumentnamessomethingthatwasneveropenedwitha
redirection,orifthereisasystemproblemclosingthefileorprocess.Inthesecases,gawksetsthepredefinedvariable
ERRNOtoastringdescribingtheproblem.
Ingawk,whenclosingapipeorcoprocess(inputoroutput),thereturnvalueistheexitstatusofthecommand.30
Otherwise,itisthereturnvaluefromthesystemsclose()orfclose()Cfunctionswhenclosinginputoroutputfiles,
respectively.Thisvalueiszeroiftheclosesucceeds,or1ifitfails.
ThePOSIXstandardisveryvagueitsaysthatclose()returnszeroonsuccessandanonzerovalueotherwise.In
general,differentimplementationsvaryinwhattheyreportwhenclosingpipesthus,thereturnvaluecannotbeused
portably.(d.c.)InPOSIXmode(seeOptions),gawkjustreturnszerowhenclosingapipe.
Next:OutputExercises,Previous:CloseFilesAndPipes,Up:Printing[Contents][Index]
100/479
29/7/2016
TheGNUAwkUsersGuide
5.10Summary
Theprintstatementprintscommaseparatedexpressions.EachexpressionisseparatedbythevalueofOFSand
terminatedbythevalueofORS.OFMTprovidestheconversionformatfornumericvaluesfortheprintstatement.
Theprintfstatementprovidesfinergrainedcontroloveroutput,withformatcontrollettersfordifferentdata
typesandvariousflagsthatmodifythebehavioroftheformatcontrolletters.
Outputfrombothprintandprintfmayberedirectedtofiles,pipes,andcoprocesses.
gawkprovidesspecialfilenamesforaccesstostandardinput,output,anderror,andfornetworkcommunications.
Useclose()tocloseopenfile,pipe,andcoprocessredirections.Forcoprocesses,itispossibletocloseonlyone
directionofthecommunications.
Previous:OutputSummary,Up:Printing[Contents][Index]
5.11Exercises
1.Rewritetheprogram:
print""}
{print$1,"",$2}'inventoryshipped
fromOutputSeparators,byusinganewvalueofOFS.
2.Usetheprintfstatementtolineuptheheadingsandtabledatafortheinventoryshippedexamplethatwas
coveredinPrint.
3.Whathappensifyouforgetthedoublequoteswhenredirectingoutput,asfollows:
BEGIN{print"Seriouserrordetected!">/dev/stderr}
Next:PatternsandActions,Previous:Printing,Up:Top[Contents][Index]
6Expressions
Expressionsarethebasicbuildingblocksofawkpatternsandactions.Anexpressionevaluatestoavaluethatyoucan
print,test,orpasstoafunction.Additionally,anexpressioncanassignanewvaluetoavariableorafieldbyusingan
assignmentoperator.
Anexpressioncanserveasapatternoractionstatementonitsown.Mostotherkindsofstatementscontainoneormore
expressionsthatspecifythedataonwhichtooperate.Asinotherlanguages,expressionsinawkcanincludevariables,
arrayreferences,constants,andfunctioncalls,aswellascombinationsofthesewithvariousoperators.
Values:
Constants,Variables,andRegularExpressions.
AllOperators:
gawksoperators.
TruthValuesandConditions: Testingfortrueandfalse.
FunctionCalls:
Afunctioncallisanexpression.
Precedence:
Howvariousoperatorsnest.
Locales:
Howthelocaleaffectsthings.
ExpressionsSummary:
Expressionssummary.
Next:AllOperators,Up:Expressions[Contents][Index]
6.1Constants,Variables,andConversions
Expressionsarebuiltupfromvaluesandtheoperationsperformeduponthem.Thissectiondescribestheelementary
objectsthatprovidethevaluesusedinexpressions.
101/479
29/7/2016
TheGNUAwkUsersGuide
Constants:
String,numericandregexpconstants.
UsingConstantRegexps: Whenandhowtousearegexpconstant.
Variables:
Variablesgivenamestovaluesforlateruse.
Conversion:
Theconversionofstringstonumbersandviceversa.
Next:UsingConstantRegexps,Up:Values[Contents][Index]
6.1.1ConstantExpressions
Thesimplesttypeofexpressionistheconstant,whichalwayshasthesamevalue.Therearethreetypesofconstants:
numeric,string,andregularexpression.
Eachisusedintheappropriatecontextwhenyouneedadatavaluethatisntgoingtochange.Numericconstantscan
havedifferentforms,butareinternallystoredinanidenticalmanner.
ScalarConstants:
Numericandstringconstants.
Nondecimalnumbers: Whatareoctalandhexnumbers.
RegexpConstants:
RegularExpressionconstants.
Next:Nondecimalnumbers,Up:Constants[Contents][Index]
6.1.1.1NumericandStringConstants
Anumericconstantstandsforanumber.Thisnumbercanbeaninteger,adecimalfraction,oranumberinscientific
(exponential)notation.31Herearesomeexamplesofnumericconstantsthatallhavethesamevalue:
105
1.05e+2
1050e1
Astringconstantconsistsofasequenceofcharactersenclosedindoublequotationmarks.Forexample:
"parrot"
representsthestringwhosecontentsareparrot.Stringsingawkcanbeofanylength,andtheycancontainanyofthe
possibleeightbitASCIIcharacters,includingASCIINUL(charactercodezero).Otherawkimplementationsmayhave
difficultywithsomecharactercodes.
Next:RegexpConstants,Previous:ScalarConstants,Up:Constants[Contents][Index]
6.1.1.2OctalandHexadecimalNumbers
Inawk,allnumbersareindecimal(i.e.,base10).Manyotherprogramminglanguagesallowyoutospecifynumbersin
otherbases,oftenoctal(base8)andhexadecimal(base16).Inoctal,thenumbersgo0,1,2,3,4,5,6,7,10,11,12,and
soon.Justas11indecimalis1times10plus1,so11inoctalis1times8plus1.Thisequals9indecimal.In
hexadecimal,thereare16digits.Becausetheeverydaydecimalnumbersystemonlyhastendigits(09),theletters
athroughfareusedtorepresenttherest.(CaseinthelettersisusuallyirrelevanthexadecimalaandAhavethe
samevalue.)Thus,11inhexadecimalis1times16plus1,whichequals17indecimal.
Justbylookingatplain11,youcanttellwhatbaseitsin.So,inC,C++,andotherlanguagesderivedfromC,thereis
aspecialnotationtosignifythebase.Octalnumbersstartwithaleading0,andhexadecimalnumbersstartwitha
leading0xor0X:
11
102/479
29/7/2016
TheGNUAwkUsersGuide
Decimalvalue11
011
Octal11,decimalvalue9
0x11
Hexadecimal11,decimalvalue17
Thisexampleshowsthedifference:
$gawk'BEGIN{printf"%d,%d,%d\n",011,11,0x11}'
|9,11,17
Beingabletouseoctalandhexadecimalconstantsinyourprogramsismostusefulwhenworkingwithdatathatcannot
berepresentedconvenientlyascharactersorasregularnumbers,suchasbinarydataofvarioussorts.
gawkallowstheuseofoctalandhexadecimalconstantsinyourprogramtext.However,suchnumbersintheinputdata
arenottreateddifferentlydoingsobydefaultwouldbreakoldprograms.(Ifyoureallyneedtodothis,usethenon
decimaldatacommandlineoptionseeNondecimalData.)Ifyouhaveoctalorhexadecimaldata,youcanusethe
strtonum()function(seeStringFunctions)toconvertthedataintoanumber.Mostofthetime,youwillwanttouse
octalorhexadecimalconstantswhenworkingwiththebuiltinbitmanipulationfunctionsseeBitwiseFunctions,for
moreinformation.
UnlikeinsomeearlyCimplementations,8and9arenotvalidinoctalconstants.Forexample,gawktreats018as
decimal18:
$gawk'BEGIN{print"021is",021;print018}'
|021is17
|18
Octalandhexadecimalsourcecodeconstantsareagawkextension.Ifgawkisincompatibilitymode(seeOptions),they
arenotavailable.
AConstantsBaseDoesNotAffectItsValue
Onceanumericconstanthasbeenconvertedinternallyintoanumber,gawknolongerrememberswhattheoriginal
formoftheconstantwastheinternalvalueisalwaysused.Thishasparticularconsequencesforconversionof
numberstostrings:
$gawk'BEGIN{printf"0x11is<%s>\n",0x11}'
|0x11is<17>
Previous:Nondecimalnumbers,Up:Constants[Contents][Index]
6.1.1.3RegularExpressionConstants
Aregexpconstantisaregularexpressiondescriptionenclosedinslashes,suchas/^beginningandend$/.Mostregexps
usedinawkprogramsareconstant,butthe~and!~matchingoperatorscanalsomatchcomputedordynamicregexps
(whicharetypicallyjustordinarystringsorvariablesthatcontainaregexp,butcouldbemorecomplexexpressions).
Next:Variables,Previous:Constants,Up:Values[Contents][Index]
6.1.2UsingRegularExpressionConstants
Whenusedontherighthandsideofthe~or!~operators,aregexpconstantmerelystandsfortheregexpthatistobe
103/479
29/7/2016
TheGNUAwkUsersGuide
matched.However,regexpconstants(suchas/foo/)maybeusedlikesimpleexpressions.Whenaregexpconstant
appearsbyitself,ithasthesamemeaningasifitappearedinapattern(i.e.,($0~/foo/)).(d.c.)SeeExpression
Patterns.Thismeansthatthefollowingtwocodesegments:
if($0~/barfly/||$0~/camelot/)
print"found"
and:
if(/barfly/||/camelot/)
print"found"
areexactlyequivalent.OneratherbizarreconsequenceofthisruleisthatthefollowingBooleanexpressionisvalid,but
doesnotdowhatitsauthorprobablyintended:
#Notethat/foo/isontheleftofthe~
if(/foo/~$1)print"foundfoo"
Thiscodeisobviouslytesting$1foramatchagainsttheregexp/foo/.Butinfact,theexpression/foo/~$1really
means($0~/foo/)~$1.Inotherwords,firstmatchtheinputrecordagainsttheregexp/foo/.Theresultiseither
zeroorone,dependinguponthesuccessorfailureofthematch.Thatresultisthenmatchedagainstthefirstfieldinthe
record.Becauseitisunlikelythatyouwouldeverreallywanttomakethiskindoftest,gawkissuesawarningwhenit
seesthisconstructinaprogram.Anotherconsequenceofthisruleisthattheassignmentstatement:
matches=/foo/
assignseitherzerooronetothevariablematches,dependinguponthecontentsofthecurrentinputrecord.
Constantregularexpressionsarealsousedasthefirstargumentforthegensub(),sub(),andgsub()functions,asthe
secondargumentofthematch()function,andasthethirdargumentofthesplit()andpatsplit()functions(seeString
Functions).Modernimplementationsofawk,includinggawk,allowthethirdargumentofsplit()tobearegexp
constant,butsomeolderimplementationsdonot.(d.c.)Becausesomebuiltinfunctionsacceptregexpconstantsas
arguments,confusioncanarisewhenattemptingtouseregexpconstantsasargumentstouserdefinedfunctions(see
Userdefined).Forexample:
functionmysub(pat,repl,str,global)
{
if(global)
gsub(pat,repl,str)
else
sub(pat,repl,str)
returnstr
}
{
text="hi!hiyourself!"
mysub(/hi/,"howdy",text,1)
Inthisexample,theprogrammerwantstopassaregexpconstanttotheuserdefinedfunctionmysub(),whichinturn
passesitontoeithersub()orgsub().However,whatreallyhappensisthatthepatparameterisassignedavalueof
eitheroneorzero,dependinguponwhetherornot$0matches/hi/.gawkissuesawarningwhenitseesaregexp
constantusedasaparametertoauserdefinedfunction,becausepassingatruthvalueinthiswayisprobablynotwhat
wasintended.
Next:Conversion,Previous:UsingConstantRegexps,Up:Values[Contents][Index]
6.1.3Variables
104/479
29/7/2016
TheGNUAwkUsersGuide
Variablesarewaysofstoringvaluesatonepointinyourprogramforuselaterinanotherpartofyourprogram.They
canbemanipulatedentirelywithintheprogramtext,andtheycanalsobeassignedvaluesontheawkcommandline.
UsingVariables: Usingvariablesinyourprograms.
Assignment
Settingvariablesonthecommandlineandasummaryofcommandlinesyntax.Thisisan
Options:
advancedmethodofinput.
Next:AssignmentOptions,Up:Variables[Contents][Index]
6.1.3.1UsingVariablesinaProgram
Variablesletyougivenamestovaluesandrefertothemlater.Variableshavealreadybeenusedinmanyofthe
examples.Thenameofavariablemustbeasequenceofletters,digits,orunderscores,anditmaynotbeginwithadigit.
Here,aletterisanyoneofthe52upperandlowercaseEnglishletters.Othercharactersthatmaybedefinedaslettersin
nonEnglishlocalesarenotvalidinvariablenames.CaseissignificantinvariablenamesaandAaredistinctvariables.
Avariablenameisavalidexpressionbyitselfitrepresentsthevariablescurrentvalue.Variablesaregivennewvalues
withassignmentoperators,incrementoperators,anddecrementoperators(seeAssignmentOps).Inaddition,thesub()
andgsub()functionscanchangeavariablesvalue,andthematch(),split(),andpatsplit()functionscanchangethe
contentsoftheirarrayparameters(seeStringFunctions).
Afewvariableshavespecialbuiltinmeanings,suchasFS(thefieldseparator)andNF(thenumberoffieldsinthe
currentinputrecord).SeeBuiltinVariables,foralistofthepredefinedvariables.Thesepredefinedvariablescanbe
usedandassignedjustlikeallothervariables,buttheirvaluesarealsousedorchangedautomaticallybyawk.All
predefinedvariablesnamesareentirelyuppercase.
Variablesinawkcanbeassignedeithernumericorstringvalues.Thekindofvalueavariableholdscanchangeoverthe
lifeofaprogram.Bydefault,variablesareinitializedtotheemptystring,whichiszeroifconvertedtoanumber.There
isnoneedtoexplicitlyinitializeavariableinawk,whichiswhatyouwoulddoinCandinmostothertraditional
languages.
Previous:UsingVariables,Up:Variables[Contents][Index]
6.1.3.2AssigningVariablesontheCommandLine
Anyawkvariablecanbesetbyincludingavariableassignmentamongtheargumentsonthecommandlinewhenawkis
invoked(seeOtherArguments).Suchanassignmenthasthefollowingform:
variable=text
Withit,avariableisseteitheratthebeginningoftheawkrunorinbetweeninputfiles.Whentheassignmentispreceded
withthevoption,asinthefollowing:
vvariable=text
thevariableissetattheverybeginning,evenbeforetheBEGINrulesexecute.Thevoptionanditsassignmentmust
precedeallthefilenamearguments,aswellastheprogramtext.(SeeOptions,formoreinformationaboutthev
option.)Otherwise,thevariableassignmentisperformedatatimedeterminedbyitspositionamongtheinputfile
argumentsaftertheprocessingoftheprecedinginputfileargument.Forexample:
awk'{print$n}'n=4inventoryshippedn=2maillist
printsthevalueoffieldnumbernforallinputrecords.Beforethefirstfileisread,thecommandlinesetsthevariablen
equaltofour.Thiscausesthefourthfieldtobeprintedinlinesfrominventoryshipped.Afterthefirstfilehasfinished,
butbeforethesecondfileisstarted,nissettotwo,sothatthesecondfieldisprintedinlinesfrommaillist:
$awk'{print$n}'n=4inventoryshippedn=2maillist
105/479
29/7/2016
TheGNUAwkUsersGuide
|15
|24
|5555553
|5553412
CommandlineargumentsaremadeavailableforexplicitexaminationbytheawkprogramintheARGVarray(seeARGC
andARGV).awkprocessesthevaluesofcommandlineassignmentsforescapesequences(seeEscapeSequences).
(d.c.)
Previous:Variables,Up:Values[Contents][Index]
6.1.4ConversionofStringsandNumbers
Numbertostringandstringtonumberconversionaregenerallystraightforward.Therecanbesubtletiestobeawareof
thissectiondiscussesthisimportantfacetofawk.
StringsAndNumbers:
HowawkConvertsBetweenStringsAndNumbers.
Localeinfluencesconversions: Howthelocalemayaffectconversions.
Next:Localeinfluencesconversions,Up:Conversion[Contents][Index]
6.1.4.1HowawkConvertsBetweenStringsandNumbers
Stringsareconvertedtonumbersandnumbersareconvertedtostrings,ifthecontextoftheawkprogramdemandsit.
Forexample,ifthevalueofeitherfooorbarintheexpressionfoo+barhappenstobeastring,itisconvertedtoa
numberbeforetheadditionisperformed.Ifnumericvaluesappearinstringconcatenation,theyareconvertedtostrings.
Considerthefollowing:
two=2;three=3
print(twothree)+4
Thisprintsthe(numeric)value27.Thenumericvaluesofthevariablestwoandthreeareconvertedtostringsand
concatenatedtogether.Theresultingstringisconvertedbacktothenumber23,towhich4isthenadded.
If,forsomereason,youneedtoforceanumbertobeconvertedtoastring,concatenatethatnumberwiththeempty
string,"".Toforceastringtobeconvertedtoanumber,addzerotothatstring.Astringisconvertedtoanumberby
interpretinganynumericprefixofthestringasnumerals:"2.5"convertsto2.5,"1e3"convertsto1,000,and"25fix"
hasanumericvalueof25.Stringsthatcantbeinterpretedasvalidnumbersconverttozero.
TheexactmannerinwhichnumbersareconvertedintostringsiscontrolledbytheawkpredefinedvariableCONVFMT(see
BuiltinVariables).Numbersareconvertedusingthesprintf()functionwithCONVFMTastheformatspecifier(seeString
Functions).
CONVFMTsdefaultvalueis"%.6g",whichcreatesavaluewithatmostsixsignificantdigits.Forsomeapplications,you
mightwanttochangeittospecifymoreprecision.Onmostmodernmachines,17digitsisusuallyenoughtocapturea
floatingpointnumbersvalueexactly.32
StrangeresultscanoccurifyousetCONVFMTtoastringthatdoesnttellsprintf()howtoformatfloatingpointnumbers
inausefulway.Forexample,ifyouforgetthe%intheformat,awkconvertsallnumberstothesameconstantstring.
Asaspecialcase,ifanumberisaninteger,thentheresultofconvertingittoastringisalwaysaninteger,nomatter
whatthevalueofCONVFMTmaybe.Giventhefollowingcodefragment:
CONVFMT="%2.2f"
a=12
b=a""
106/479
29/7/2016
TheGNUAwkUsersGuide
bhasthevalue"12",not"12.00".(d.c.)
PrePOSIXawkUsedOFMTforStringConversion
PriortothePOSIXstandard,awkusedthevalueofOFMTforconvertingnumberstostrings.OFMTspecifiestheoutput
formattousewhenprintingnumberswithprint.CONVFMTwasintroducedinordertoseparatethesemanticsof
conversionfromthesemanticsofprinting.BothCONVFMTandOFMThavethesamedefaultvalue:"%.6g".Inthevast
majorityofcases,oldawkprogramsdonotchangetheirbehavior.SeePrint,formoreinformationontheprint
statement.
Previous:StringsAndNumbers,Up:Conversion[Contents][Index]
6.1.4.2LocalesCanInfluenceConversion
Whereyouarecanmatterwhenitcomestoconvertingbetweennumbersandstrings.Thelocalcharactersetand
languagethelocalecanaffectnumericformats.Inparticular,forawkprograms,itaffectsthedecimalpointcharacter
andthethousandsseparatorcharacter.The"C"locale,andmostEnglishlanguagelocales,usetheperiodcharacter(.)
asthedecimalpointanddonthaveathousandsseparator.However,many(ifnotmost)EuropeanandnonEnglish
localesusethecomma(,)asthedecimalpointcharacter.Europeanlocalesoftenuseeitheraspaceoraperiodasthe
thousandsseparator,iftheyhaveone.
ThePOSIXstandardsaysthatawkalwaysusestheperiodasthedecimalpointwhenreadingtheawkprogramsource
code,andforcommandlinevariableassignments(seeOtherArguments).However,wheninterpretinginputdata,for
printandprintfoutput,andfornumbertostringconversion,thelocaldecimalpointcharacterisused.(d.c.)Inall
cases,numbersinsourcecodeandininputdatacannothaveathousandsseparator.Herearesomeexamplesindicating
thedifferenceinbehavior,onaGNU/Linuxsystem:
$exportPOSIXLY_CORRECT=1ForcePOSIXbehavior
$gawk'BEGIN{printf"%g\n",3.1415927}'
|3.14159
$LC_ALL=en_DK.utf8gawk'BEGIN{printf"%g\n",3.1415927}'
|3,14159
$echo4,321|gawk'{print$1+1}'
|5
$echo4,321|LC_ALL=en_DK.utf8gawk'{print$1+1}'
|5,321
Theen_DK.utf8localeisforEnglishinDenmark,wherethecommaactsasthedecimalpointseparator.Inthenormal
"C"locale,gawktreats4,321as4,whileintheDanishlocale,itstreatedasthefullnumberincludingthefractional
part,4.321.
Someearlierversionsofgawkfullycompliedwiththisaspectofthestandard.However,manyusersinnonEnglish
localescomplainedaboutthisbehavior,becausetheirdatausedaperiodasthedecimalpoint,sothedefaultbehavior
wasrestoredtouseaperiodasthedecimalpointcharacter.Youcanusetheuselcnumericoption(seeOptions)to
forcegawktousethelocalesdecimalpointcharacter.(gawkalsousesthelocalesdecimalpointcharacterwhenin
POSIXmode,eitherviaposixorthePOSIXLY_CORRECTenvironmentvariable,asshownpreviously.)
Table6.1describesthecasesinwhichthelocalesdecimalpointcharacterisusedandwhenaperiodisused.Someof
thesefeatureshavenotbeendescribedyet.
Feature
%'g
%g
Input
strtonum()
Default
Uselocale
Useperiod
Useperiod
Useperiod
posixoruselcnumeric
Uselocale
Uselocale
Uselocale
Uselocale
107/479
29/7/2016
TheGNUAwkUsersGuide
Table6.1:Localedecimalpointversusaperiod
Finally,moderndayformalstandardsandtheIEEEstandardfloatingpointrepresentationcanhaveanunusualbut
importanteffectonthewaygawkconvertssomespecialstringvaluestonumbers.ThedetailsarepresentedinPOSIX
FloatingPointProblems.
Next:TruthValuesandConditions,Previous:Values,Up:Expressions[Contents][Index]
6.2Operators:DoingSomethingwithValues
Thissectionintroducestheoperatorsthatmakeuseofthevaluesprovidedbyconstantsandvariables.
ArithmeticOps: Arithmeticoperations(+,,etc.)
Concatenation: Concatenatingstrings.
AssignmentOps: Changingthevalueofavariableorafield.
IncrementOps: Incrementingthenumericvalueofavariable.
Next:Concatenation,Up:AllOperators[Contents][Index]
6.2.1ArithmeticOperators
Theawklanguageusesthecommonarithmeticoperatorswhenevaluatingexpressions.Allofthesearithmeticoperators
follownormalprecedencerulesandworkasyouwouldexpectthemto.
Thefollowingexampleusesafilenamedgrades,whichcontainsalistofstudentnamesaswellasthreetestscoresper
student(itsasmallclass):
Pat1009758
Sandy847293
Chris729289
Thisprogramtakesthefilegradesandprintstheaverageofthescores:
$awk'{sum=$2+$3+$4;avg=sum/3
>print$1,avg}'grades
|Pat85
|Sandy83
|Chris84.3333
Thefollowinglistprovidesthearithmeticoperatorsinawk,inorderfromthehighestprecedencetothelowest:
x^y
x**y
Exponentiationxraisedtotheypower.2^3hasthevalueeightthecharactersequence**isequivalentto
^.(c.e.)
x
Negation.
+x
Unaryplustheexpressionisconvertedtoanumber.
x*y
Multiplication.
108/479
29/7/2016
TheGNUAwkUsersGuide
x/y
Divisionbecauseallnumbersinawkarefloatingpointnumbers,theresultisnotroundedtoaninteger3/4
hasthevalue0.75.(Itisacommonmistake,especiallyforCprogrammers,toforgetthatallnumbersinawkare
floatingpoint,andthatdivisionofintegerlookingconstantsproducesarealnumber,notaninteger.)
x%y
Remainderfurtherdiscussionisprovidedinthetext,justafterthislist.
x+y
Addition.
xy
Subtraction.
Unaryplusandminushavethesameprecedence,themultiplicationoperatorsallhavethesameprecedence,and
additionandsubtractionhavethesameprecedence.
Whencomputingtheremainderofx%y,thequotientisroundedtowardzerotoanintegerandmultipliedbyy.This
resultissubtractedfromxthisoperationissometimesknownastruncmod.Thefollowingrelationalwaysholds:
b*int(a/b)+(a%b)==a
Onepossiblyundesirableeffectofthisdefinitionofremainderisthatx%yisnegativeifxisnegative.Thus:
17%8=1
Inotherawkimplementations,thesignednessoftheremaindermaybemachinedependent.
NOTE:ThePOSIXstandardonlyspecifiestheuseof^forexponentiation.Formaximumportability,do
notusethe**operator.
Next:AssignmentOps,Previous:ArithmeticOps,Up:AllOperators[Contents][Index]
6.2.2StringConcatenation
Itseemedlikeagoodideaatthetime.
BrianKernighan
Thereisonlyonestringoperation:concatenation.Itdoesnothaveaspecificoperatortorepresentit.Instead,
concatenationisperformedbywritingexpressionsnexttooneanother,withnooperator.Forexample:
$awk'{print"Fieldnumberone:"$1}'maillist
|Fieldnumberone:Amelia
|Fieldnumberone:Anthony
Withoutthespaceinthestringconstantafterthe:,thelinerunstogether.Forexample:
$awk'{print"Fieldnumberone:"$1}'maillist
|Fieldnumberone:Amelia
|Fieldnumberone:Anthony
Becausestringconcatenationdoesnothaveanexplicitoperator,itisoftennecessarytoensurethatithappensatthe
righttimebyusingparenthesestoenclosetheitemstoconcatenate.Forexample,youmightexpectthatthefollowing
codefragmentconcatenatesfileandname:
109/479
29/7/2016
TheGNUAwkUsersGuide
file="file"
name="name"
print"somethingmeaningful">filename
ThisproducesasyntaxerrorwithsomeversionsofUnixawk.33Itisnecessarytousethefollowing:
print"somethingmeaningful">(filename)
Parenthesesshouldbeusedaroundconcatenationinallbutthemostcommoncontexts,suchasontherighthandsideof
=.Becarefulaboutthekindsofexpressionsusedinstringconcatenation.Inparticular,theorderofevaluationof
expressionsusedforconcatenationisundefinedintheawklanguage.Considerthisexample:
BEGIN{
a="don't"
print(a""(a="panic"))
}
Itisnotdefinedwhetherthesecondassignmenttoahappensbeforeorafterthevalueofaisretrievedforproducingthe
concatenatedvalue.Theresultcouldbeeitherdon'tpanic,orpanicpanic.
Theprecedenceofconcatenation,whenmixedwithotheroperators,isoftencounterintuitive.Considerthisexample:
$awk'BEGIN{print12""24}'
|1224
Thisobviouslyisconcatenating12,aspace,and24.Butwheredidthespacedisappearto?Theanswerliesinthe
combinationofoperatorprecedencesandawksautomaticconversionrules.Togetthedesiredresult,writetheprogram
thisway:
$awk'BEGIN{print12""(24)}'
|1224
Thisforcesawktotreattheonthe24asunary.Otherwise,itsparsedasfollows:
12(""24)
12(024)
12(24)
1224
Asmentionedearlier,whenmixingconcatenationwithotheroperators,parenthesize.Otherwise,youreneverquitesure
whatyoullget.
Next:IncrementOps,Previous:Concatenation,Up:AllOperators[Contents][Index]
6.2.3AssignmentExpressions
Anassignmentisanexpressionthatstoresa(usuallydifferent)valueintoavariable.Forexample,letsassignthevalue
onetothevariablez:
z=1
Afterthisexpressionisexecuted,thevariablezhasthevalueone.Whateveroldvaluezhadbeforetheassignmentis
forgotten.
Assignmentscanalsostorestringvalues.Forexample,thefollowingstoresthevalue"thisfoodisgood"inthe
variablemessage:
thing="food"
predicate="good"
message="this"thing"is"predicate
110/479
29/7/2016
TheGNUAwkUsersGuide
Thisalsoillustratesstringconcatenation.The=signiscalledanassignmentoperator.Itisthesimplestassignment
operatorbecausethevalueoftherighthandoperandisstoredunchanged.Mostoperators(addition,concatenation,and
soon)havenoeffectexcepttocomputeavalue.Ifthevalueisntused,theresnoreasontousetheoperator.An
assignmentoperatorisdifferentitdoesproduceavalue,butevenifyouignoreit,theassignmentstillmakesitselffelt
throughthealterationofthevariable.Wecallthisasideeffect.
Thelefthandoperandofanassignmentneednotbeavariable(seeVariables)itcanalsobeafield(seeChanging
Fields)oranarrayelement(seeArrays).Theseareallcalledlvalues,whichmeanstheycanappearonthelefthandside
ofanassignmentoperator.Therighthandoperandmaybeanyexpressionitproducesthenewvaluethattheassignment
storesinthespecifiedvariable,field,orarrayelement.(Suchvaluesarecalledrvalues.)
Itisimportanttonotethatvariablesdonothavepermanenttypes.Avariablestypeissimplythetypeofwhatevervalue
waslastassignedtoit.Inthefollowingprogramfragment,thevariablefoohasanumericvalueatfirst,andastring
valuelateron:
foo=1
printfoo
foo="bar"
printfoo
Whenthesecondassignmentgivesfooastringvalue,thefactthatitpreviouslyhadanumericvalueisforgotten.
Stringvaluesthatdonotbeginwithadigithaveanumericvalueofzero.Afterexecutingthefollowingcode,thevalue
offooisfive:
foo="astring"
foo=foo+5
NOTE:Usingavariableasanumberandthenlaterasastringcanbeconfusingandispoorprogramming
style.Theprevioustwoexamplesillustratehowawkworks,nothowyoushouldwriteyourprograms!
Anassignmentisanexpression,soithasavaluethesamevaluethatisassigned.Thus,z=1isanexpressionwith
thevalueone.Oneconsequenceofthisisthatyoucanwritemultipleassignmentstogether,suchas:
x=y=z=5
Thisexamplestoresthevaluefiveinallthreevariables(x,y,andz).Itdoessobecausethevalueofz=5,whichis
five,isstoredintoyandthenthevalueofy=z=5,whichisfive,isstoredintox.
Assignmentsmaybeusedanywhereanexpressioniscalledfor.Forexample,itisvalidtowritex!=(y=1)tosety
toone,andthentestwhetherxequalsone.Butthisstyletendstomakeprogramshardtoreadsuchnestingof
assignmentsshouldbeavoided,exceptperhapsinaoneshotprogram.
Asidefrom=,thereareseveralotherassignmentoperatorsthatdoarithmeticwiththeoldvalueofthevariable.For
example,theoperator+=computesanewvaluebyaddingtherighthandvaluetotheoldvalueofthevariable.Thus,
thefollowingassignmentaddsfivetothevalueoffoo:
foo+=5
Thisisequivalenttothefollowing:
foo=foo+5
Usewhichevermakesthemeaningofyourprogramclearer.
Therearesituationswhereusing+=(oranyassignmentoperator)isnotthesameassimplyrepeatingthelefthand
operandintherighthandexpression.Forexample:
#ThankstoPatRankinforthisexample
BEGIN{
foo[rand()]+=5
for(xinfoo)
111/479
29/7/2016
TheGNUAwkUsersGuide
printx,foo[x]
bar[rand()]=bar[rand()]+5
for(xinbar)
printx,bar[x]
}
Theindicesofbararepracticallyguaranteedtobedifferent,becauserand()returnsdifferentvalueseachtimeitis
called.(Arraysandtherand()functionhaventbeencoveredyet.SeeArrays,andseeNumericFunctions,formore
information.)Thisexampleillustratesanimportantfactaboutassignmentoperators:thelefthandexpressionisonly
evaluatedonce.
Itisuptotheimplementationastowhichexpressionisevaluatedfirst,thelefthandortherighthand.Considerthis
example:
i=1
a[i+=2]=i+1
Thevalueofa[3]couldbeeithertwoorfour.
Table6.2liststhearithmeticassignmentoperators.Ineachcase,therighthandoperandisanexpressionwhosevalueis
convertedtoanumber.
Operator
lvalue+=increment
lvalue=decrement
lvalue*=coefficient
lvalue/=divisor
lvalue%=modulus
lvalue^=power
lvalue**=power
Effect
Addincrementtothevalueoflvalue.
Subtractdecrementfromthevalueoflvalue.
Multiplythevalueoflvaluebycoefficient.
Dividethevalueoflvaluebydivisor.
Setlvaluetoitsremainderbymodulus.
Raiselvaluetothepowerpower.
Raiselvaluetothepowerpower.(c.e.)
Table6.2:Arithmeticassignmentoperators
NOTE:Onlythe^=operatorisspecifiedbyPOSIX.Formaximumportability,donotusethe**=
operator.
SyntacticAmbiguitiesBetween/=andRegularExpressions
Thereisasyntacticambiguitybetweenthe/=assignmentoperatorandregexpconstantswhosefirstcharacterisan=.
(d.c.)Thisismostnotableinsomecommercialawkversions.Forexample:
$awk/==//dev/null
errorawk:syntaxerroratsourceline1
errorcontextis
error>>>/=<<<
errorawk:bailingoutatsourceline1
Aworkaroundis:
awk'/[=]=/'/dev/null
gawkdoesnothavethisproblemBWKawkandmawkalsodonot.
Previous:AssignmentOps,Up:AllOperators[Contents][Index]
6.2.4IncrementandDecrementOperators
112/479
29/7/2016
TheGNUAwkUsersGuide
Incrementanddecrementoperatorsincreaseordecreasethevalueofavariablebyone.Anassignmentoperatorcando
thesamething,sotheincrementoperatorsaddnopowertotheawklanguagehowever,theyareconvenient
abbreviationsforverycommonoperations.
Theoperatorusedforaddingoneiswritten++.Itcanbeusedtoincrementavariableeitherbeforeoraftertakingits
value.Topreincrementavariablev,write++v.Thisaddsonetothevalueofvthatnewvalueisalsothevalueofthe
expression.(Theassignmentexpressionv+=1iscompletelyequivalent.)Writingthe++afterthevariablespecifies
postincrement.Thisincrementsthevariablevaluejustthesamethedifferenceisthatthevalueoftheincrement
expressionitselfisthevariablesoldvalue.Thus,iffoohasthevaluefour,thentheexpressionfoo++hasthevalue
four,butitchangesthevalueoffootofive.Inotherwords,theoperatorreturnstheoldvalueofthevariable,butwith
thesideeffectofincrementingit.
Thepostincrementfoo++isnearlythesameaswriting(foo+=1)1.Itisnotperfectlyequivalentbecauseall
numbersinawkarefloatingpointinfloatingpoint,foo+11doesnotnecessarilyequalfoo.Butthedifferenceis
minuteaslongasyousticktonumbersthatarefairlysmall(lessthan10e12).
Fieldsandarrayelementsareincrementedjustlikevariables.(Use$(i++)whenyouwanttodoafieldreferenceanda
variableincrementatthesametime.Theparenthesesarenecessarybecauseoftheprecedenceofthefieldreference
operator$.)
Thedecrementoperatorworksjustlike++,exceptthatitsubtractsoneinsteadofaddingit.Aswith++,itcanbe
usedbeforethelvaluetopredecrementorafterittopostdecrement.Followingisasummaryofincrementand
decrementexpressions:
++lvalue
Incrementlvalue,returningthenewvalueasthevalueoftheexpression.
lvalue++
Incrementlvalue,returningtheoldvalueoflvalueasthevalueoftheexpression.
lvalue
Decrementlvalue,returningthenewvalueasthevalueoftheexpression.(Thisexpressionislike++lvalue,but
insteadofadding,itsubtracts.)
lvalue
Decrementlvalue,returningtheoldvalueoflvalueasthevalueoftheexpression.(Thisexpressionislike
lvalue++,butinsteadofadding,itsubtracts.)
OperatorEvaluationOrder
Doctor,ithurtswhenIdothis!
Thendontdothat!
GrouchoMarx
Whathappensforsomethinglikethefollowing?
b=6
printb+=b++
Orsomethingevenstranger?
b=6
b+=++b+b++
printb
Inotherwords,whendothevarioussideeffectsprescribedbythepostfixoperators(b++)takeeffect?Whenside
113/479
29/7/2016
TheGNUAwkUsersGuide
effectshappenisimplementationdefined.Inotherwords,itisuptotheparticularversionofawk.Theresultforthe
firstexamplemaybe12or13,andforthesecond,itmaybe22or23.
Inshort,doingthingslikethisisnotrecommendedanddefinitelynotanythingthatyoucanrelyuponforportability.
Youshouldavoidsuchthingsinyourownprograms.
Next:FunctionCalls,Previous:AllOperators,Up:Expressions[Contents][Index]
6.3TruthValuesandConditions
Incertaincontexts,expressionvaluesalsoserveastruthvaluesi.e.,theydeterminewhatshouldhappennextasthe
programruns.Thissectiondescribeshowawkdefinestrueandfalseandhowvaluesarecompared.
TruthValues:
Typingand
Comparison:
BooleanOps:
ConditionalExp:
Whatistrueandwhatisfalse.
Howvariablesacquiretypesandhowthisaffectscomparisonofnumbersandstringswith<,
etc.
Combiningcomparisonexpressionsusingbooleanoperators||(or),&&(and)and!
(not).
Conditionalexpressionsselectbetweentwosubexpressionsundercontrolofathird
subexpression.
Next:TypingandComparison,Up:TruthValuesandConditions[Contents][Index]
6.3.1TrueandFalseinawk
Manyprogramminglanguageshaveaspecialrepresentationfortheconceptsoftrueandfalse.Suchlanguages
usuallyusethespecialconstantstrueandfalse,orperhapstheiruppercaseequivalents.However,awkisdifferent.It
borrowsaverysimpleconceptoftrueandfalsefromC.Inawk,anynonzeronumericvalueoranynonemptystring
valueistrue.Anyothervalue(zeroorthenullstring,"")isfalse.ThefollowingprogramprintsAstrangetruth
valuethreetimes:
BEGIN{
if(3.1415927)
print"Astrangetruthvalue"
if("FourScoreAndSevenYearsAgo")
if(j=57)
}
Thereisasurprisingconsequenceofthenonzeroornonnullrule:thestringconstant"0"isactuallytrue,becauseitis
nonnull.(d.c.)
Next:BooleanOps,Previous:TruthValues,Up:TruthValuesandConditions[Contents][Index]
6.3.2VariableTypingandComparisonExpressions
TheGuideisdefinitive.Realityisfrequentlyinaccurate.
DouglasAdams,TheHitchhikersGuidetotheGalaxy
Unlikeinotherprogramminglanguages,inawkvariablesdonothaveafixedtype.Instead,theycanbeeitheranumber
orastring,dependinguponthevaluethatisassignedtothem.Welooknowathowvariablesaretyped,andhowawk
comparesvariables.
114/479
29/7/2016
TheGNUAwkUsersGuide
VariableTyping:
Stringtypeversusnumerictype.
ComparisonOperators:
Thecomparisonoperators.
POSIXStringComparison: StringcomparisonwithPOSIXrules.
Next:ComparisonOperators,Up:TypingandComparison[Contents][Index]
6.3.2.1StringTypeversusNumericType
ThePOSIXstandardintroducedtheconceptofanumericstring,whichissimplyastringthatlookslikeanumberfor
example,"+2".Thisconceptisusedfordeterminingthetypeofavariable.Thetypeofthevariableisimportant
becausethetypesoftwovariablesdeterminehowtheyarecompared.Variabletypingfollowstheserules:
Anumericconstantortheresultofanumericoperationhasthenumericattribute.
Astringconstantortheresultofastringoperationhasthestringattribute.
Fields,getlineinput,FILENAME,ARGVelements,ENVIRONelements,andtheelementsofanarraycreatedby
match(),split(),andpatsplit()thatarenumericstringshavethestrnumattribute.Otherwise,theyhavethe
stringattribute.Uninitializedvariablesalsohavethestrnumattribute.
Attributespropagateacrossassignmentsbutarenotchangedbyanyuse.
Thelastruleisparticularlyimportant.Inthefollowingprogram,ahasnumerictype,eventhoughitislaterusedina
stringoperation:
BEGIN{
a=12.345
b=a"isacutenumber"
printb
}
Whentwooperandsarecompared,eitherstringcomparisonornumericcomparisonmaybeused.Thisdependsuponthe
attributesoftheoperands,accordingtothefollowingsymmetricmatrix:
+
|STRINGNUMERICSTRNUM
+
|
STRING|stringstringstring
|
NUMERIC|stringnumericnumeric
|
STRNUM|stringnumericnumeric
+
Thebasicideaisthatuserinputthatlooksnumericandonlyuserinputshouldbetreatedasnumeric,eventhoughit
isactuallymadeofcharactersandisthereforealsoastring.Thus,forexample,thestringconstant"+3.14",whenit
appearsinprogramsourcecode,isastringeventhoughitlooksnumericandisnevertreatedasanumberfor
comparisonpurposes.
Inshort,whenoneoperandisapurestring,suchasastringconstant,thenastringcomparisonisperformed.
Otherwise,anumericcomparisonisperformed.
Thispointbearsadditionalemphasis:Alluserinputismadeofcharacters,andsoisfirstandforemostofstringtype
inputstringsthatlooknumericareadditionallygiventhestrnumattribute.Thus,thesixcharacterinputstring+3.14
receivesthestrnumattribute.Incontrast,theeightcharacters"+3.14"appearinginprogramtextcompriseastring
constant.Thefollowingexamplesprint1whenthecomparisonbetweenthetwodifferentconstantsistrue,and0
otherwise:
$echo'+3.14'|awk'{print($0=="+3.14")}'True
|1
$echo'+3.14'|awk'{print($0=="+3.14")}'False
115/479
29/7/2016
TheGNUAwkUsersGuide
|0
$echo'+3.14'|awk'{print($0=="3.14")}'False
|0
$echo'+3.14'|awk'{print($0==3.14)}'True
|1
$echo'+3.14'|awk'{print($1=="+3.14")}'False
|0
$echo'+3.14'|awk'{print($1=="+3.14")}'True
|1
$echo'+3.14'|awk'{print($1=="3.14")}'False
|0
$echo'+3.14'|awk'{print($1==3.14)}'True
|1
Next:POSIXStringComparison,Previous:VariableTyping,Up:TypingandComparison[Contents][Index]
6.3.2.2ComparisonOperators
Comparisonexpressionscomparestringsornumbersforrelationshipssuchasequality.Theyarewrittenusingrelational
operators,whichareasupersetofthoseinC.Table6.3describesthem.
Expression
x<y
x<=y
x>y
x>=y
x==y
x!=y
x~y
x!~y
subscriptinarray
Result
Trueifxislessthany
Trueifxislessthanorequaltoy
Trueifxisgreaterthany
Trueifxisgreaterthanorequaltoy
Trueifxisequaltoy
Trueifxisnotequaltoy
Trueifthestringxmatchestheregexpdenotedbyy
Trueifthestringxdoesnotmatchtheregexpdenotedbyy
Trueifthearrayarrayhasanelementwiththesubscript
subscript
Table6.3:Relationaloperators
Comparisonexpressionshavethevalueoneiftrueandzeroiffalse.Whencomparingoperandsofmixedtypes,numeric
operandsareconvertedtostringsusingthevalueofCONVFMT(seeConversion).
Stringsarecomparedbycomparingthefirstcharacterofeach,thenthesecondcharacterofeach,andsoon.Thus,"10"
islessthan"9".Iftherearetwostringswhereoneisaprefixoftheother,theshorterstringislessthanthelongerone.
Thus,"abc"islessthan"abcd".
Itisveryeasytoaccidentallymistypethe==operatorandleaveoffoneofthe=characters.Theresultisstillvalid
awkcode,buttheprogramdoesnotdowhatisintended:
if(a=b)#oops!shouldbea==b
else
Unlessbhappenstobezeroorthenullstring,theifpartofthetestalwayssucceeds.Becausetheoperatorsareso
similar,thiskindoferrorisverydifficulttospotwhenscanningthesourcecode.
Thefollowinglistofexpressionsillustratesthekindsofcomparisonsawkperforms,aswellaswhattheresultofeach
comparisonis:
1.5<=2.0
116/479
29/7/2016
TheGNUAwkUsersGuide
Numericcomparison(true)
"abc">="xyz"
Stringcomparison(false)
1.5!="+2"
Stringcomparison(true)
"1e2"<"3"
a=2;b="2"
a==b
a=2;b="+2"
a==b
Stringcomparison(false)
Inthisexample:
$echo1e23|awk'{print($1<$2)?"true":"false"}'
|false
theresultisfalsebecauseboth$1and$2areuserinput.Theyarenumericstringsthereforebothhavethestrnum
attribute,dictatinganumericcomparison.Thepurposeofthecomparisonrulesandtheuseofnumericstringsisto
attempttoproducethebehaviorthatisleastsurprising,whilestilldoingtherightthing.
Stringcomparisonsandregularexpressioncomparisonsareverydifferent.Forexample:
x=="foo"
hasthevalueone,oristrueifthevariablexispreciselyfoo.Bycontrast:
x~/foo/
hasthevalueoneifxcontainsfoo,suchas"Oh,whatafoolamI!".
Therighthandoperandofthe~and!~operatorsmaybeeitheraregexpconstant(//)oranordinaryexpression.In
thelattercase,thevalueoftheexpressionasastringisusedasadynamicregexp(seeRegexpUsagealsosee
ComputedRegexps).
Aconstantregularexpressioninslashesbyitselfisalsoanexpression./regexp/isanabbreviationforthefollowing
comparisonexpression:
$0~/regexp/
Onespecialplacewhere/foo/isnotanabbreviationfor$0~/foo/iswhenitistherighthandoperandof~or!~.
SeeUsingConstantRegexps,wherethisisdiscussedinmoredetail.
Previous:ComparisonOperators,Up:TypingandComparison[Contents][Index]
6.3.2.3StringComparisonwithPOSIXRules
ThePOSIXstandardsaysthatstringcomparisonisperformedbasedonthelocalescollatingorder.Thisistheorderin
whichcharacterssort,asdefinedbythelocale(formorediscussion,seeLocales).Thisorderisusuallyverydifferent
117/479
29/7/2016
TheGNUAwkUsersGuide
fromtheresultsobtainedwhendoingstraightcharacterbycharactercomparison.34
Becausethisbehaviordiffersconsiderablyfromexistingpractice,gawkonlyimplementsitwheninPOSIXmode(see
Options).Hereisanexampletoillustratethedifference,inanen_US.UTF8locale:
$gawk'BEGIN{printf("ABC<abc=%s\n",
>("ABC"<"abc"?"TRUE":"FALSE"))}'
|ABC<abc=TRUE
$gawkposix'BEGIN{printf("ABC<abc=%s\n",
>("ABC"<"abc"?"TRUE":"FALSE"))}'
|ABC<abc=FALSE
Next:ConditionalExp,Previous:TypingandComparison,Up:TruthValuesandConditions[Contents][Index]
6.3.3BooleanExpressions
ABooleanexpressionisacombinationofcomparisonexpressionsormatchingexpressions,usingtheBooleanoperators
or(||),and(&&),andnot(!),alongwithparenthesestocontrolnesting.ThetruthvalueoftheBoolean
expressioniscomputedbycombiningthetruthvaluesofthecomponentexpressions.Booleanexpressionsarealso
referredtoaslogicalexpressions.Thetermsareequivalent.
Booleanexpressionscanbeusedwherevercomparisonandmatchingexpressionscanbeused.Theycanbeusedinif,
while,do,andforstatements(seeStatements).Theyhavenumericvalues(oneiftrue,zeroiffalse)thatcomeintoplay
iftheresultoftheBooleanexpressionisstoredinavariableorusedinarithmetic.
Inaddition,everyBooleanexpressionisalsoavalidpattern,soyoucanuseoneasapatterntocontroltheexecutionof
rules.TheBooleanoperatorsare:
boolean1&&boolean2
Trueifbothboolean1andboolean2aretrue.Forexample,thefollowingstatementprintsthecurrentinputrecord
ifitcontainsbotheduandli:
if($0~/edu/&&$0~/li/)print
Thesubexpressionboolean2isevaluatedonlyifboolean1istrue.Thiscanmakeadifferencewhenboolean2
containsexpressionsthathavesideeffects.Inthecaseof$0~/foo/&&($2==bar++),thevariablebarisnot
incrementedifthereisnosubstringfoointherecord.
boolean1||boolean2
Trueifatleastoneofboolean1orboolean2istrue.Forexample,thefollowingstatementprintsallrecordsinthe
inputthatcontaineithereduorli:
if($0~/edu/||$0~/li/)print
Thesubexpressionboolean2isevaluatedonlyifboolean1isfalse.Thiscanmakeadifferencewhenboolean2
containsexpressionsthathavesideeffects.(Thus,thistestneverreallydistinguishesrecordsthatcontainboth
eduandliassoonaseduismatched,thefulltestsucceeds.)
!boolean
Trueifbooleanisfalse.Forexample,thefollowingprogramprintsnohome!intheunusualeventthattheHOME
environmentvariableisnotdefined:
BEGIN{if(!("HOME"inENVIRON))
print"nohome!"}
(TheinoperatorisdescribedinReferencetoElements.)
118/479
29/7/2016
TheGNUAwkUsersGuide
The&&and||operatorsarecalledshortcircuitoperatorsbecauseofthewaytheywork.Evaluationofthefull
expressionisshortcircuitediftheresultcanbedeterminedpartwaythroughitsevaluation.
Statementsthatendwith&&or||canbecontinuedsimplybyputtinganewlineafterthem.Butyoucannotputa
newlineinfrontofeitheroftheseoperatorswithoutusingbackslashcontinuation(seeStatements/Lines).
Theactualvalueofanexpressionusingthe!operatoriseitheroneorzero,dependinguponthetruthvalueofthe
expressionitisappliedto.The!operatorisoftenusefulforchangingthesenseofaflagvariablefromfalsetotrueand
backagain.Forexample,thefollowingprogramisonewaytoprintlinesinbetweenspecialbracketinglines:
$1=="START"{interested=!interested;next}
interested{print}
$1=="END"{interested=!interested;next}
Thevariableinterested,aswithallawkvariables,startsoutinitializedtozero,whichisalsofalse.Whenalineisseen
whosefirstfieldisSTART,thevalueofinterestedistoggledtotrue,using!.Thenextruleprintslinesaslongas
interestedistrue.WhenalineisseenwhosefirstfieldisEND,interestedistoggledbacktofalse.35
Mostcommonly,the!operatorisusedintheconditionsofifandwhilestatements,whereitoftenmakesmoresense
tophrasethelogicinthenegative:
if(!somecondition||someothercondition){
dowhateverprocessing
}
NOTE:ThenextstatementisdiscussedinNextStatement.nexttellsawktoskiptherestoftherules,get
thenextrecord,andstartprocessingtherulesoveragainatthetop.Thereasonitsthereistoavoidprinting
thebracketingSTARTandENDlines.
Previous:BooleanOps,Up:TruthValuesandConditions[Contents][Index]
6.3.4ConditionalExpressions
Aconditionalexpressionisaspecialkindofexpressionthathasthreeoperands.Itallowsyoutouseoneexpressions
valuetoselectoneoftwootherexpressions.TheconditionalexpressioninawkisthesameasintheClanguage,as
shownhere:
selector?iftrueexp:iffalseexp
Therearethreesubexpressions.Thefirst,selector,isalwayscomputedfirst.Ifitistrue(notzeroornotnull),thenif
trueexpiscomputednext,anditsvaluebecomesthevalueofthewholeexpression.Otherwise,iffalseexpiscomputed
next,anditsvaluebecomesthevalueofthewholeexpression.Forexample,thefollowingexpressionproducesthe
absolutevalueofx:
x>=0?x:x
Eachtimetheconditionalexpressioniscomputed,onlyoneofiftrueexpandiffalseexpisusedtheotherisignored.
Thisisimportantwhentheexpressionshavesideeffects.Forexample,thisconditionalexpressionexamineselementi
ofeitherarrayaorarrayb,andincrementsi:
x==y?a[i++]:b[i++]
Thisisguaranteedtoincrementiexactlyonce,becauseeachtimeonlyoneofthetwoincrementexpressionsis
executedandtheotherisnot.SeeArrays,formoreinformationaboutarrays.
Asaminorgawkextension,astatementthatuses?:canbecontinuedsimplybyputtinganewlineaftereither
character.However,puttinganewlineinfrontofeithercharacterdoesnotworkwithoutusingbackslashcontinuation
(seeStatements/Lines).Ifposixisspecified(seeOptions),thisextensionisdisabled.
119/479
29/7/2016
TheGNUAwkUsersGuide
Next:Precedence,Previous:TruthValuesandConditions,Up:Expressions[Contents][Index]
6.4FunctionCalls
Afunctionisanameforaparticularcalculation.Thisenablesyoutoaskforitbynameatanypointintheprogram.For
example,thefunctionsqrt()computesthesquarerootofanumber.
Afixedsetoffunctionsarebuiltin,whichmeanstheyareavailableineveryawkprogram.Thesqrt()functionisoneof
these.SeeBuiltin,foralistofbuiltinfunctionsandtheirdescriptions.Inaddition,youcandefinefunctionsforusein
yourprogram.SeeUserdefined,forinstructionsonhowtodothis.Finally,gawkletsyouwritefunctionsinCorC++
thatmaybecalledfromyourprogram(seeDynamicExtensions).
Thewaytouseafunctioniswithafunctioncallexpression,whichconsistsofthefunctionnamefollowedimmediately
byalistofargumentsinparentheses.Theargumentsareexpressionsthatprovidetherawmaterialsforthefunctions
calculations.Whenthereismorethanoneargument,theyareseparatedbycommas.Iftherearenoarguments,justwrite
()afterthefunctionname.Thefollowingexamplesshowfunctioncallswithandwithoutarguments:
sqrt(x^2+y^2)oneargument
atan2(y,x)twoarguments
rand()noarguments
CAUTION:Donotputanyspacebetweenthefunctionnameandtheopeningparenthesis!Auserdefined
functionnamelooksjustlikethenameofavariableaspacewouldmaketheexpressionlooklike
concatenationofavariablewithanexpressioninsideparentheses.Withbuiltinfunctions,spacebeforethe
parenthesisisharmless,butitisbestnottogetintothehabitofusingspacetoavoidmistakeswithuser
definedfunctions.
Eachfunctionexpectsaparticularnumberofarguments.Forexample,thesqrt()functionmustbecalledwithasingle
argument,thenumberofwhichtotakethesquareroot:
sqrt(argument)
Someofthebuiltinfunctionshaveoneormoreoptionalarguments.Ifthoseargumentsarenotsupplied,thefunctions
useareasonabledefaultvalue.SeeBuiltin,forfulldetails.Ifargumentsareomittedincallstouserdefinedfunctions,
thenthoseargumentsaretreatedaslocalvariables.Suchlocalvariablesactliketheemptystringifreferencedwherea
stringvalueisrequired,andlikezeroifreferencedwhereanumericvalueisrequired(seeUserdefined).
Asanadvancedfeature,gawkprovidesindirectfunctioncalls,whichisawaytochoosethefunctiontocallatruntime,
insteadofwhenyouwritethesourcecodetoyourprogram.WedeferdiscussionofthisfeatureuntillaterseeIndirect
Calls.
Likeeveryotherexpression,thefunctioncallhasavalue,oftencalledthereturnvalue,whichiscomputedbythe
functionbasedontheargumentsyougiveit.Inthisexample,thereturnvalueofsqrt(argument)isthesquarerootof
argument.Thefollowingprogramreadsnumbers,onenumberperline,andprintsthesquarerootofeachone:
$awk'{print"Thesquarerootof",$1,"is",sqrt($1)}'
1
|Thesquarerootof1is1
3
|Thesquarerootof3is1.73205
5
|Thesquarerootof5is2.23607
Ctrld
Afunctioncanalsohavesideeffects,suchasassigningvaluestocertainvariablesordoingI/O.Thisprogramshows
howthematch()function(seeStringFunctions)changesthevariablesRSTARTandRLENGTH:
{
if(match($1,$2))
printRSTART,RLENGTH
else
120/479
29/7/2016
TheGNUAwkUsersGuide
print"nomatch"
}
Hereisasamplerun:
$awkfmatchit.awk
aaccddc+
|32
foobar
|nomatch
abcdefge
|51
Next:Locales,Previous:FunctionCalls,Up:Expressions[Contents][Index]
6.5OperatorPrecedence(HowOperatorsNest)
Operatorprecedencedetermineshowoperatorsaregroupedwhendifferentoperatorsappearclosebyinoneexpression.
Forexample,*hashigherprecedencethan+thus,a+b*cmeanstomultiply bandc,andthenaddatothe
product(i.e.,a+(b*c)).
Thenormalprecedenceoftheoperatorscanbeoverruledbyusingparentheses.Thinkoftheprecedencerulesassaying
wheretheparenthesesareassumedtobe.Infact,itiswisetoalwaysuseparentheseswheneverthereisanunusual
combinationofoperators,becauseotherpeoplewhoreadtheprogrammaynotrememberwhattheprecedenceisinthis
case.Evenexperiencedprogrammersoccasionallyforgettheexactrules,whichleadstomistakes.Explicitparentheses
helppreventanysuchmistakes.
Whenoperatorsofequalprecedenceareusedtogether,theleftmostoperatorgroupsfirst,exceptfortheassignment,
conditional,andexponentiationoperators,whichgroupintheoppositeorder.Thus,ab+cgroupsas(ab)+c
anda=b=cgroupsasa=(b=c).
Normallytheprecedenceofprefixunaryoperatorsdoesnotmatter,becausethereisonlyonewaytointerpretthem:
innermostfirst.Thus,$++imeans$(++i)and++$xmeans++($x).However,whenanotheroperatorfollowsthe
operand,thentheprecedenceoftheunaryoperatorscanmatter.$x^2means($x)^2,butx^2means(x^2),
becausehaslowerprecedencethan^,whereas$hashigherprecedence.Also,operatorscannotbecombinedina
waythatviolatestheprecedencerulesforexample,$$0++isnotavalidexpressionbecausethefirst$hashigher
precedencethanthe++toavoidtheproblemtheexpressioncanberewrittenas$($0++).
Thislistpresentsawksoperators,inorderofhighesttolowestprecedence:
()
Grouping.
$
Fieldreference.
++
Increment,decrement.
^**
Exponentiation.Theseoperatorsgrouprighttoleft.
+!
Unaryplus,minus,logicalnot.
*/%
121/479
29/7/2016
TheGNUAwkUsersGuide
Multiplication,division,remainder.
+
Addition,subtraction.
Stringconcatenation
Thereisnospecialsymbolforconcatenation.Theoperandsaresimplywrittensidebyside(seeConcatenation).
<<===!=>>=>>||&
Relationalandredirection.Therelationaloperatorsandtheredirectionshavethesameprecedencelevel.
Characterssuchas>servebothasrelationalsandasredirectionsthecontextdistinguishesbetweenthetwo
meanings.
NotethattheI/Oredirectionoperatorsinprintandprintfstatementsbelongtothestatementlevel,notto
expressions.Theredirectiondoesnotproduceanexpressionthatcouldbetheoperandofanotheroperator.Asa
result,itdoesnotmakesensetousearedirectionoperatornearanotheroperatoroflowerprecedencewithout
parentheses.Suchcombinations(e.g.,printfoo>a?b:c)resultinsyntaxerrors.Thecorrectwaytowrite
thisstatementisprintfoo>(a?b:c).
~!~
Matching,nonmatching.
in
Arraymembership.
&&
Logicaland.
||
Logicalor.
?:
Conditional.Thisoperatorgroupsrighttoleft.
=+==*=/=%=^=**=
Assignment.Theseoperatorsgrouprighttoleft.
NOTE:The|&,**,and**=operatorsarenotspecifiedbyPOSIX.Formaximumportability,donot
usethem.
Next:ExpressionsSummary,Previous:Precedence,Up:Expressions[Contents][Index]
6.6WhereYouAreMakesaDifference
Modernsystemssupportthenotionoflocales:awaytotellthesystemaboutthelocalcharactersetandlanguage.The
ISOCstandarddefinesadefault"C"locale,whichisanenvironmentthatistypicalofwhatmanyCprogrammersare
usedto.
Onceuponatime,thelocalesettingusedtoaffectregexpmatching,butthisisnolongertrue(seeRangesandLocales).
Localescanaffectrecordsplitting.ForthenormalcaseofRS="\n",thelocaleislargelyirrelevant.Forothersingle
122/479
29/7/2016
TheGNUAwkUsersGuide
characterrecordseparators,settingLC_ALL=Cintheenvironmentwillgiveyoumuchbetterperformancewhenreading
records.Otherwise,gawkhastomakeseveralfunctioncalls,perinputcharacter,tofindtherecordterminator.
Localescanaffecthowdatesandtimesareformatted(seeTimeFunctions).Forexample,acommonwaytoabbreviate
thedateSeptember4,2015,intheUnitedStatesis9/4/15.InmanycountriesinEurope,however,itisabbreviated
4.9.15.Thus,the%xspecificationina "US"localemightproduce9/4/15,whileina"EUROPE"locale,itmight
produce4.9.15.
AccordingtoPOSIX,stringcomparisonisalsoaffectedbylocales(similartoregularexpressions).Thedetailsare
presentedinPOSIXStringComparison.
Finally,thelocaleaffectsthevalueofthedecimalpointcharacterusedwhengawkparsesinputdata.Thisisdiscussedin
detailinConversion.
Previous:Locales,Up:Expressions[Contents][Index]
6.7Summary
Expressionsarethebasicelementsofcomputationinprograms.Theyarebuiltfromconstants,variables,function
calls,andcombinationsofthevariouskindsofvalueswithoperators.
awksuppliesthreekindsofconstants:numeric,string,andregexp.gawkletsyouspecifynumericconstantsinoctal
andhexadecimal(bases8and16)aswellasdecimal(base10).Incertaincontexts,astandaloneregexpconstant
suchas/foo/hasthesamemeaningas$0~/foo/.
Variablesholdvaluesbetweenusesincomputations.Anumberofbuiltinvariablesprovideinformationtoyour
awkprogram,andanumberofothersletyoucontrolhowawkbehaves.
Numbersareautomaticallyconvertedtostrings,andstringstonumbers,asneededbyawk.Numericvaluesare
convertedasiftheywereformattedwithsprintf()usingtheformatinCONVFMT.Localescaninfluencethe
conversions.
awkprovidestheusualarithmeticoperators(addition,subtraction,multiplication,division,modulus),andunary
plusandminus.Italsoprovidescomparisonoperators,Booleanoperators,anarraymembershiptestingoperator,
andregexpmatchingoperators.Stringconcatenationisaccomplishedbyplacingtwoexpressionsnexttoeach
otherthereisnoexplicitoperator.Thethreeoperand?:operatorprovidesanifelsetestwithinexpressions.
Assignmentoperatorsprovideconvenientshorthandsforcommonarithmeticoperations.
Inawk,avalueisconsideredtobetrueifitisnonzeroornonnull.Otherwise,thevalueisfalse.
Avariablestypeissetuponeachassignmentandmaychangeoveritslifetime.Thetypedetermineshowit
behavesincomparisons(stringornumeric).
Functioncallsreturnavaluethatmaybeusedaspartofalargerexpression.Expressionsusedtopassparameter
valuesarefullyevaluatedbeforethefunctioniscalled.awkprovidesbuiltinanduserdefinedfunctionsthisis
describedinFunctions.
Operatorprecedencespecifiestheorderinwhichoperationsareperformed,unlessexplicitlyoverriddenby
parentheses.awksoperatorprecedenceiscompatiblewiththatofC.
Localescanaffecttheformatofdataasoutputbyanawkprogram,andoccasionallytheformatfordatareadas
input.
Next:Arrays,Previous:Expressions,Up:Top[Contents][Index]
Asyouhavealreadyseen,eachawkstatementconsistsofapatternwithanassociatedaction.Thischapterdescribeshow
youbuildpatternsandactions,whatkindsofthingsyoucandowithinactions,andawkspredefinedvariables.
Thepatternactionrulesandthestatementsavailableforusewithinactionsformthecoreofawkprogramming.Ina
sense,everythingcovereduptoherehasbeenthefoundationthatprogramsarebuiltontopof.Nowitstimetostart
buildingsomethinguseful.
123/479
29/7/2016
TheGNUAwkUsersGuide
PatternOverview:
Whatgoesintoapattern.
UsingShellVariables: Howtouseshellvariableswithawk.
ActionOverview:
Whatgoesintoanaction.
Statements:
Describesthevariouscontrolstatementsindetail.
BuiltinVariables:
Summarizesthepredefinedvariables.
PatternActionSummary: PatternsandActionssummary.
Next:UsingShellVariables,Up:PatternsandActions[Contents][Index]
7.1PatternElements
RegexpPatterns:
Usingregexpsaspatterns.
ExpressionPatterns:
Anyexpressioncanbeusedasapattern.
Ranges:
Pairsofpatternsspecifyrecordranges.
BEGIN/END:
Specifyinginitializationandcleanuprules.
BEGINFILE/ENDFILE: Twospecialpatternsforadvancedcontrol.
Empty:
Theemptypattern,whichmatcheseveryrecord.
Patternsinawkcontroltheexecutionofrulesaruleisexecutedwhenitspatternmatchesthecurrentinputrecord.The
followingisasummaryofthetypesofawkpatterns:
/regularexpression/
Aregularexpression.Itmatcheswhenthetextoftheinputrecordfitstheregularexpression.(SeeRegexp.)
expression
Asingleexpression.Itmatcheswhenitsvalueisnonzero(ifanumber)ornonnull(ifastring).(SeeExpression
Patterns.)
begpat,endpat
Apairofpatternsseparatedbyacomma,specifyingarangeofrecords.Therangeincludesboththeinitialrecord
thatmatchesbegpatandthefinalrecordthatmatchesendpat.(SeeRanges.)
BEGIN
END
Specialpatternsforyoutosupplystartuporcleanupactionsforyourawkprogram.(SeeBEGIN/END.)
BEGINFILE
ENDFILE
Specialpatternsforyoutosupplystartuporcleanupactionstobedoneonaperfilebasis.(See
BEGINFILE/ENDFILE.)
empty
Theemptypatternmatcheseveryinputrecord.(SeeEmpty.)
Next:ExpressionPatterns,Up:PatternOverview[Contents][Index]
7.1.1RegularExpressionsasPatterns
Regularexpressionsareoneofthefirstkindsofpatternspresentedinthisbook.Thiskindofpatternissimplyaregexp
constantinthepatternpartofarule.Itsmeaningis$0~/pattern/.Thepatternmatcheswhentheinputrecord
matchestheregexp.Forexample:
124/479
29/7/2016
TheGNUAwkUsersGuide
/foo|bar|baz/{buzzwords++}
END{printbuzzwords,"buzzwordsseen"}
Next:Ranges,Previous:RegexpPatterns,Up:PatternOverview[Contents][Index]
7.1.2ExpressionsasPatterns
Anyawkexpressionisvalidasanawkpattern.Thepatternmatchesiftheexpressionsvalueisnonzero(ifanumber)or
nonnull(ifastring).Theexpressionisreevaluatedeachtimetheruleistestedagainstanewinputrecord.Ifthe
expressionusesfieldssuchas$1,thevaluedependsdirectlyonthenewinputrecordstextotherwise,itdependson
onlywhathashappenedsofarintheexecutionoftheawkprogram.
Comparisonexpressions,usingthecomparisonoperatorsdescribedinTypingandComparison,areaverycommonkind
ofpattern.Regexpmatchingandnonmatchingarealsoverycommonexpressions.Theleftoperandofthe~and!~
operatorsisastring.Therightoperandiseitheraconstantregularexpressionenclosedinslashes(/regexp/),orany
expressionwhosestringvalueisusedasadynamicregularexpression(seeComputedRegexps).Thefollowingexample
printsthesecondfieldofeachinputrecordwhosefirstfieldispreciselyli:
$awk'$1=="li"{print$2}'maillist
(Thereisnooutput,becausethereisnopersonwiththeexactnameli.)Contrastthiswiththefollowingregular
expressionmatch,whichacceptsanyrecordwithafirstfieldthatcontainsli:
$awk'$1~/li/{print$2}'maillist
|5555553
|5556699
Aregexpconstantasapatternisalsoaspecialcaseofanexpressionpattern.Theexpression/li/hasthevalueoneif
liappearsinthecurrentinputrecord.Thus,asapattern, /li/matchesanyrecordcontainingli.
Booleanexpressionsarealsocommonlyusedaspatterns.Whetherthepatternmatchesaninputrecorddependson
whetheritssubexpressionsmatch.Forexample,thefollowingcommandprintsalltherecordsinmaillistthatcontain
botheduandli:
$awk'/edu/&&/li/'maillist
Thefollowingcommandprintsallrecordsinmaillistthatcontaineithereduorli(orboth,ofcourse):
$awk'/edu/||/li/'maillist
|Broderick5550542broderick.aliquotiens@yahoo.comR
Thefollowingcommandprintsallrecordsinmaillistthatdonotcontainthestringli:
$awk'!/li/'maillist
|Anthony5553412anthony.asserturo@hotmail.comA
|Becky5557685becky.algebrarum@gmail.comA
|Bill5551675bill.drowning@hotmail.comA
|Camilla5552912camilla.infusarum@skynet.beR
|Martin5556480martin.codicibus@hotmail.comA
ThesubexpressionsofaBooleanoperatorinapatterncanbeconstantregularexpressions,comparisons,oranyother
awkexpressions.Rangepatternsarenotexpressions,sotheycannotappearinsideBooleanpatterns.Likewise,the
specialpatternsBEGIN,END,BEGINFILE,andENDFILE,whichnevermatchanyinputrecord,arenotexpressionsandcannot
appearinsideBooleanpatterns.
125/479
29/7/2016
TheGNUAwkUsersGuide
TheprecedenceofthedifferentoperatorsthatcanappearinpatternsisdescribedinPrecedence.
Next:BEGIN/END,Previous:ExpressionPatterns,Up:PatternOverview[Contents][Index]
7.1.3SpecifyingRecordRangeswithPatterns
Arangepatternismadeoftwopatternsseparatedbyacomma,intheformbegpat,endpat.Itisusedtomatchranges
ofconsecutiveinputrecords.Thefirstpattern,begpat,controlswheretherangebegins,whileendpatcontrolswherethe
patternends.Forexample,thefollowing:
awk'$1=="on",$1=="off"'myfile
printseveryrecordinmyfilebetweenon/offpairs,inclusive.
Arangepatternstartsoutbymatchingbegpatagainsteveryinputrecord.Whenarecordmatchesbegpat,therange
patternisturnedon,andtherangepatternmatchesthisrecordaswell.Aslongastherangepatternstaysturnedon,it
automaticallymatcheseveryinputrecordread.Therangepatternalsomatchesendpatagainsteveryinputrecordwhen
thissucceeds,therangepatternisturnedoffagainforthefollowingrecord.Thentherangepatterngoesbackto
checkingbegpatagainsteachrecord.
Therecordthatturnsontherangepatternandtheonethatturnsitoffbothmatchtherangepattern.Ifyoudontwantto
operateontheserecords,youcanwriteifstatementsintherulesactiontodistinguishthemfromtherecordsyouare
interestedin.
Itispossibleforapatterntobeturnedonandoffbythesamerecord.Iftherecordsatisfiesbothconditions,thenthe
actionisexecutedforjustthatrecord.Forexample,supposethereistextbetweentwoidenticalmarkers(e.g.,the%
symbol),eachonitsownline,thatshouldbeignored.Afirstattemptwouldbetocombinearangepatternthatdescribes
thedelimitedtextwiththenextstatement(notdiscussedyet,seeNextStatement).Thiscausesawktoskipanyfurther
processingofthecurrentrecordandstartoveragainwiththenextinputrecord.Suchaprogramlookslikethis:
/^%$/,/^%$/{next}
{print}
Thisprogramfailsbecausetherangepatternisbothturnedonandturnedoffbythefirstline,whichjusthasa%onit.
Toaccomplishthistask,writetheprograminthefollowingmanner,usingaflag:
/^%$/{skip=!skip;next}
skip==1{next}#skiplineswith`skip'set
Inarangepattern,thecomma(,)hasthelowestprecedenceofalltheoperators(i.e.,itisevaluatedlast).Thus,the
followingprogramattemptstocombinearangepatternwithanother,simplertest:
echoYes|awk'/1/,/2/||/Yes/'
Theintentofthisprogramis(/1/,/2/)||/Yes/.However,awkinterpretsthisas/1/,(/2/||/Yes/).Thiscannot
bechangedorworkedaroundrangepatternsdonotcombinewithotherpatterns:
$echoYes|gawk'(/1/,/2/)||/Yes/'
errorgawk:cmd.line:1:(/1/,/2/)||/Yes/
errorgawk:cmd.line:1:^syntaxerror
Asaminorpointofinterest,althoughitispoorstyle,POSIXallowsyoutoputanewlineafterthecommainarange
pattern.(d.c.)
Next:BEGINFILE/ENDFILE,Previous:Ranges,Up:PatternOverview[Contents][Index]
7.1.4TheBEGINandENDSpecialPatterns
126/479
29/7/2016
TheGNUAwkUsersGuide
Allthepatternsdescribedsofarareformatchinginputrecords.TheBEGINandENDspecialpatternsaredifferent.They
supplystartupandcleanupactionsforawkprograms.BEGINandENDrulesmusthaveactionsthereisnodefaultaction
fortheserulesbecausethereisnocurrentrecordwhentheyrun.BEGINandENDrulesareoftenreferredtoasBEGINand
ENDblocksbylongtimeawkprogrammers.
UsingBEGIN/END: HowandwhytouseBEGIN/ENDrules.
I/OAndBEGIN/END: I/OissuesinBEGIN/ENDrules.
Next:I/OAndBEGIN/END,Up:BEGIN/END[Contents][Index]
7.1.4.1StartupandCleanupActions
ABEGINruleisexecutedonceonly,beforethefirstinputrecordisread.Likewise,anENDruleisexecutedonceonly,
afteralltheinputisread.Forexample:
$awk'
>BEGIN{print"Analysisof\"li\""}
>/li/{++n}
>END{print"\"li\"appearsin",n,"records."}'maillist
|Analysisof"li"
|"li"appearsin4records.
Thisprogramfindsthenumberofrecordsintheinputfilemaillistthatcontainthestringli.TheBEGINruleprintsa
titleforthereport.ThereisnoneedtousetheBEGINruletoinitializethecounterntozero,asawkdoesthisautomatically
(seeVariables).Thesecondruleincrementsthevariableneverytimearecordcontainingthepatternliisread.The
ENDruleprintsthevalueofnattheendoftherun.
ThespecialpatternsBEGINandENDcannotbeusedinrangesorwithBooleanoperators(indeed,theycannotbeusedwith
anyoperators).AnawkprogrammayhavemultipleBEGINand/orENDrules.Theyareexecutedintheorderinwhichthey
appear:alltheBEGINrulesatstartupandalltheENDrulesattermination.BEGINandENDrulesmaybeintermixedwith
otherrules.Thisfeaturewasaddedinthe1987versionofawkandisincludedinthePOSIXstandard.Theoriginal
(1978)versionofawkrequiredtheBEGINruletobeplacedatthebeginningoftheprogram,theENDruletobeplacedat
theend,andonlyallowedoneofeach.Thisisnolongerrequired,butitisagoodideatofollowthistemplateintermsof
programorganizationandreadability.
MultipleBEGINandENDrulesareusefulforwritinglibraryfunctions,becauseeachlibraryfilecanhaveitsownBEGIN
and/orENDruletodoitsowninitializationand/orcleanup.Theorderinwhichlibraryfunctionsarenamedonthe
commandlinecontrolstheorderinwhichtheirBEGINandENDrulesareexecuted.Therefore,youhavetobecareful
whenwritingsuchrulesinlibraryfilessothattheorderinwhichtheyareexecuteddoesntmatter.SeeOptions,for
moreinformationonusinglibraryfunctions.SeeLibraryFunctions,foranumberofusefullibraryfunctions.
IfanawkprogramhasonlyBEGINrulesandnootherrules,thentheprogramexitsaftertheBEGINrulesarerun.36
However,ifanENDruleexists,thentheinputisread,eveniftherearenootherrulesintheprogram.Thisisnecessaryin
casetheENDrulecheckstheFNRandNRvariables.
Previous:UsingBEGIN/END,Up:BEGIN/END[Contents][Index]
7.1.4.2Input/OutputfromBEGINandENDRules
Thereareseveral(sometimessubtle)pointstobeawareofwhendoingI/OfromaBEGINorENDrule.Thefirsthastodo
withthevalueof$0inaBEGINrule.BecauseBEGINrulesareexecutedbeforeanyinputisread,theresimplyisnoinput
record,andthereforenofields,whenexecutingBEGINrules.Referencesto$0andthefieldsyieldanullstringorzero,
dependinguponthecontext.Onewaytogive$0arealvalueistoexecuteagetlinecommandwithoutavariable(see
Getline).Anotherwayissimplytoassignavalueto$0.
Thesecondpointissimilartothefirst,butfromtheotherdirection.Traditionally,duelargelytoimplementationissues,
$0andNFwereundefinedinsideanENDrule.ThePOSIXstandardspecifiesthatNFisavailableinanENDrule.Itcontains
127/479
29/7/2016
TheGNUAwkUsersGuide
thenumberoffieldsfromthelastinputrecord.Mostprobablyduetoanoversight,thestandarddoesnotsaythat$0is
alsopreserved,althoughlogicallyonewouldthinkthatitshouldbe.Infact,allofBWKawk,mawk,andgawkpreservethe
valueof$0foruseinENDrules.Beaware,however,thatsomeotherimplementationsandmanyolderversionsofUnix
awkdonot.
Thethirdpointfollowsfromthefirsttwo.Themeaningofprintinsidea BEGINorENDruleisthesameasalways:
print$0.If$0isthenullstring,thenthisprintsanemptyrecord.Manylongtimeawkprogrammersuseanunadorned
printinBEGINandENDrules,tomeanprint"",relyingon$0beingnull.Althoughonemightgenerallygetaway
withthisinBEGINrules,itisaverybadideainENDrules,atleastingawk.Itisalsopoorstyle,becauseifanemptylineis
neededintheoutput,theprogramshouldprintoneexplicitly.
Finally,thenextandnextfilestatementsarenotallowedinaBEGINrule,becausetheimplicitreadarecordandmatch
againsttherulesloophasnotstartedyet.Similarly,thosestatementsarenotvalidinanENDrule,becausealltheinput
hasbeenread.(SeeNextStatement,andseeNextfileStatement,.)
Next:Empty,Previous:BEGIN/END,Up:PatternOverview[Contents][Index]
7.1.5TheBEGINFILEandENDFILESpecialPatterns
Thissectiondescribesagawkspecificfeature.
Twospecialkindsofrule,BEGINFILEandENDFILE,giveyouhooksintogawkscommandlinefileprocessingloop.As
withtheBEGINandENDrules(seeBEGIN/END),allBEGINFILErulesinaprogramaremerged,intheordertheyareread
bygawk,andallENDFILErulesaremergedaswell.
ThebodyoftheBEGINFILErulesisexecutedjustbeforegawkreadsthefirstrecordfromafile.FILENAMEissettothe
nameofthecurrentfile,andFNRissettozero.
TheBEGINFILEruleprovidesyoutheopportunitytoaccomplishtwotasksthatwouldotherwisebedifficultorimpossible
toperform:
Youcantestifthefileisreadable.Normally,itisafatalerrorifafilenamedonthecommandlinecannotbe
openedforreading.However,youcanbypassthefatalerrorandmoveontothenextfileonthecommandline.
YoudothisbycheckingiftheERRNOvariableisnottheemptystringifso,thengawkwasnotabletoopenthefile.
Inthiscase,yourprogramcanexecutethenextfilestatement(seeNextfileStatement).Thiscausesgawktoskip
thefileentirely.Otherwise,gawkexitswiththeusualfatalerror.
Ifyouhavewrittenextensionsthatmodifytherecordhandling(byinsertinganinputparserseeInputParsers),
youcaninvokethematthispoint,beforegawkhasstartedprocessingthefile.(Thisisaveryadvancedfeature,
currentlyusedonlybythegawkextlibproject.)
TheENDFILEruleiscalledwhengawkhasfinishedprocessingthelastrecordinaninputfile.Forthelastinputfile,itwill
becalledbeforeanyENDrules.TheENDFILEruleisexecutedevenforemptyinputfiles.
Normally,whenanerroroccurswhenreadinginputinthenormalinputprocessingloop,theerrorisfatal.However,if
anENDFILEruleispresent,theerrorbecomesnonfatal,andinsteadERRNOisset.Thismakesitpossibletocatchand
processI/Oerrorsattheleveloftheawkprogram.
Thenextstatement(seeNextStatement)isnotallowedinsideeitheraBEGINFILEoranENDFILErule.Thenextfile
statementisallowedonlyinsideaBEGINFILErule,notinsideanENDFILErule.
Thegetlinestatement(seeGetline)isrestrictedinsidebothBEGINFILEandENDFILE:onlyredirectedformsofgetline
areallowed.
BEGINFILEandENDFILEaregawkextensions.Inmostotherawkimplementations,orifgawkisincompatibilitymode(see
Options),theyarenotspecial.
128/479
29/7/2016
TheGNUAwkUsersGuide
Previous:BEGINFILE/ENDFILE,Up:PatternOverview[Contents][Index]
7.1.6TheEmptyPattern
Anempty(i.e.,nonexistent)patternisconsideredtomatcheveryinputrecord.Forexample,theprogram:
awk'{print$1}'maillist
printsthefirstfieldofeveryrecord.
Next:ActionOverview,Previous:PatternOverview,Up:PatternsandActions[Contents][Index]
7.2UsingShellVariablesinPrograms
awkprogramsareoftenusedascomponentsinlargerprogramswritteninshell.Forexample,itisverycommontousea
shellvariabletoholdapatternthattheawkprogramsearchesfor.Therearetwowaystogetthevalueoftheshell
variableintothebodyoftheawkprogram.
Acommonmethodistouseshellquotingtosubstitutethevariablesvalueintotheprograminsidethescript.For
example,considerthefollowingprogram:
printf"Entersearchpattern:"
readpattern
awk"/$pattern/"'{nmatches++}
END{printnmatches,"found"}'/path/to/data
Theawkprogramconsistsoftwopiecesofquotedtextthatareconcatenatedtogethertoformtheprogram.Thefirstpart
isdoublequoted,whichallowssubstitutionofthepatternshellvariableinsidethequotes.Thesecondpartissingle
quoted.
Variablesubstitutionviaquotingworks,butcanpotentiallybemessy.Itrequiresagoodunderstandingoftheshells
quotingrules(seeQuoting),anditsoftendifficulttocorrectlymatchupthequoteswhenreadingtheprogram.
Abettermethodistouseawksvariableassignmentfeature(seeAssignmentOptions)toassigntheshellvariablesvalue
toanawkvariable.Thenusedynamicregexpstomatchthepattern(seeComputedRegexps).Thefollowingshowshow
toredothepreviousexampleusingthistechnique:
printf"Entersearchpattern:"
readpattern
awkvpat="$pattern"'$0~pat{nmatches++}
END{printnmatches,"found"}'/path/to/data
Now,theawkprogramisjustonesinglequotedstring.Theassignmentvpat="$pattern"stillrequiresdoublequotes,
incasethereiswhitespaceinthevalueof$pattern.Theawkvariablepatcouldbenamedpatterntoo,butthatwouldbe
moreconfusing.Usingavariablealsoprovidesmoreflexibility,asthevariablecanbeusedanywhereinsidethe
programforprinting,asanarraysubscript,orforanyotherusewithoutrequiringthequotingtricksateverypointin
theprogram.
Next:Statements,Previous:UsingShellVariables,Up:PatternsandActions[Contents][Index]
7.3Actions
Anawkprogramorscriptconsistsofaseriesofrulesandfunctiondefinitionsinterspersed.(Functionsaredescribed
later.SeeUserdefined.)Arulecontainsapatternandanaction,eitherofwhich(butnotboth)maybeomitted.The
purposeoftheactionistotellawkwhattodoonceamatchforthepatternisfound.Thus,inoutline,anawkprogram
generallylookslikethis:
129/479
29/7/2016
TheGNUAwkUsersGuide
[pattern]{action}
pattern[{action}]
functionname(args){}
Anactionconsistsofoneormoreawkstatements,enclosedinbraces({}).Eachstatementspecifiesonethingtodo.
Thestatementsareseparatedbynewlinesorsemicolons.Thebracesaroundanactionmustbeusedeveniftheaction
containsonlyonestatement,orifitcontainsnostatementsatall.However,ifyouomittheactionentirely,omitthe
bracesaswell.Anomittedactionisequivalentto{print$0}:
/foo/{}matchfoo,donothingemptyaction
/foo/matchfoo,printtherecordomittedaction
Thefollowingtypesofstatementsaresupportedinawk:
Expressions
Callfunctionsorassignvaluestovariables(seeExpressions).Executingthiskindofstatementsimplycomputes
thevalueoftheexpression.Thisisusefulwhentheexpressionhassideeffects(seeAssignmentOps).
Controlstatements
Specifythecontrolflowofawkprograms.TheawklanguagegivesyouClikeconstructs(if,for,while,anddo)as
wellasafewspecialones(seeStatements).
Compoundstatements
Encloseoneormorestatementsinbraces.Acompoundstatementisusedinordertoputseveralstatements
togetherinthebodyofanif,while,do,orforstatement.
Inputstatements
Usethegetlinecommand(seeGetline).Alsosuppliedinawkarethenextstatement(seeNextStatement)andthe
nextfilestatement(seeNextfileStatement).
Outputstatements
Suchasprintandprintf.SeePrinting.
Deletionstatements
Fordeletingarrayelements.SeeDelete.
Next:BuiltinVariables,Previous:ActionOverview,Up:PatternsandActions[Contents][Index]
7.4ControlStatementsinActions
Controlstatements,suchasif,while,andsoon,controltheflowofexecutioninawkprograms.Mostofawkscontrol
statementsarepatternedaftersimilarstatementsinC.
Allthecontrolstatementsstartwithspecialkeywords,suchasifandwhile,todistinguishthemfromsimple
expressions.Manycontrolstatementscontainotherstatements.Forexample,theifstatementcontainsanother
statementthatmayormaynotbeexecuted.Thecontainedstatementiscalledthebody.Toincludemorethanone
statementinthebody,groupthemintoasinglecompoundstatementwithbraces,separatingthemwithnewlinesor
semicolons.
IfStatement:
Conditionallyexecutesomeawkstatements.
130/479
29/7/2016
TheGNUAwkUsersGuide
WhileStatement: Loopuntilsomeconditionissatisfied.
DoStatement:
Dospecifiedactionwhileloopinguntilsomeconditionissatisfied.
ForStatement:
Anotherloopingstatement,thatprovidesinitializationandincrementclauses.
SwitchStatement: Switch/caseevaluationforconditionalexecutionofstatementsbasedonavalue.
BreakStatement: Immediatelyexittheinnermostenclosingloop.
ContinueStatement: Skiptotheendoftheinnermostenclosingloop.
NextStatement:
Stopprocessingthecurrentinputrecord.
NextfileStatement: Stopprocessingthecurrentfile.
ExitStatement:
Stopexecutionofawk.
Next:WhileStatement,Up:Statements[Contents][Index]
7.4.1TheifelseStatement
Theifelsestatementisawksdecisionmakingstatement.Itlookslikethis:
if(condition)thenbody[elseelsebody]
Theconditionisanexpressionthatcontrolswhattherestofthestatementdoes.Iftheconditionistrue,thenbodyis
executedotherwise,elsebodyisexecuted.Theelsepartofthestatementisoptional.Theconditionisconsideredfalse
ifitsvalueiszeroorthenullstringotherwise,theconditionistrue.Refertothefollowing:
if(x%2==0)
print"xiseven"
else
print"xisodd"
Inthisexample,iftheexpressionx%2==0istrue(i.e.,ifthevalueofxisevenlydivisiblebytwo),thenthefirst
printstatementisexecutedotherwise,thesecondprintstatementisexecuted.Iftheelsekeywordappearsonthesame
lineasthenbodyandthenbodyisnotacompoundstatement(i.e.,notsurroundedbybraces),thenasemicolonmust
separatethenbodyfromtheelse.Toillustratethis,thepreviousexamplecanberewrittenas:
if(x%2==0)print"xiseven";else
print"xisodd"
Ifthe;isleftout,awkcantinterpretthestatementanditproducesasyntaxerror.Dontactuallywriteprogramsthis
way,becauseahumanreadermightfailtoseetheelseifitisnotthefirstthingonitsline.
Next:DoStatement,Previous:IfStatement,Up:Statements[Contents][Index]
7.4.2ThewhileStatement
Inprogramming,aloopisapartofaprogramthatcanbeexecutedtwoormoretimesinsuccession.Thewhile
statementisthesimplestloopingstatementinawk.Itrepeatedlyexecutesastatementaslongasaconditionistrue.For
example:
while(condition)
body
bodyisastatementcalledthebodyoftheloop,andconditionisanexpressionthatcontrolshowlongtheloopkeeps
running.Thefirstthingthewhilestatementdoesistestthecondition.Iftheconditionistrue,itexecutesthestatement
body.Afterbodyhasbeenexecuted,conditionistestedagain,andifitisstilltrue,bodyexecutesagain.Thisprocess
repeatsuntiltheconditionisnolongertrue.Iftheconditionisinitiallyfalse,thebodyoftheloopneverexecutesand
awkcontinueswiththestatementfollowingtheloop.Thisexampleprintsthefirstthreefieldsofeachrecord,oneper
line:
awk'
131/479
29/7/2016
TheGNUAwkUsersGuide
{
i=1
while(i<=3){
print$i
i++
}
}'inventoryshipped
Thebodyofthisloopisacompoundstatementenclosedinbraces,containingtwostatements.Theloopworksinthe
followingmanner:first,thevalueofiissettoone.Then,thewhilestatementtestswhetheriislessthanorequalto
three.Thisistruewheniequalsone,sotheithfieldisprinted.Thenthei++incrementsthevalueofiandtheloop
repeats.Theloopterminateswhenireachesfour.
Anewlineisnotrequiredbetweentheconditionandthebodyhowever,usingonemakestheprogramclearerunlessthe
bodyisacompoundstatementorelseisverysimple.Thenewlineaftertheopenbracethatbeginsthecompound
statementisnotrequiredeither,buttheprogramishardertoreadwithoutit.
Next:ForStatement,Previous:WhileStatement,Up:Statements[Contents][Index]
7.4.3ThedowhileStatement
Thedoloopisavariationofthewhileloopingstatement.Thedoloopexecutesthebodyonceandthenrepeatsthebody
aslongastheconditionistrue.Itlookslikethis:
do
body
while(condition)
Eveniftheconditionisfalseatthestart,thebodyexecutesatleastonce(andonlyonce,unlessexecutingbodymakes
conditiontrue).Contrastthiswiththecorrespondingwhilestatement:
while(condition)
body
Thisstatementdoesnotexecutethebodyevenonceiftheconditionisfalsetobeginwith.Thefollowingisanexample
ofadostatement:
{
i=1
do{
print$0
i++
}while(i<=10)
}
Thisprogramprintseachinputrecord10times.However,itisntaveryrealisticexample,becauseinthiscasean
ordinarywhilewoulddojustaswell.Thissituationreflectsactualexperienceonlyoccasionallyistherearealusefora
dostatement.
Next:SwitchStatement,Previous:DoStatement,Up:Statements[Contents][Index]
7.4.4TheforStatement
Theforstatementmakesitmoreconvenienttocountiterationsofaloop.Thegeneralformoftheforstatementlooks
likethis:
for(initialization;condition;increment)
body
Theinitialization,condition,andincrementpartsarearbitraryawkexpressions,andbodystandsforanyawkstatement.
132/479
29/7/2016
TheGNUAwkUsersGuide
Theforstatementstartsbyexecutinginitialization.Then,aslongastheconditionistrue,itrepeatedlyexecutesbody
andthenincrement.Typically,initializationsetsavariabletoeitherzeroorone,incrementaddsonetoit,andcondition
comparesitagainstthedesirednumberofiterations.Forexample:
awk'
{
for(i=1;i<=3;i++)
print$i
}'inventoryshipped
Thisprintsthefirstthreefieldsofeachinputrecord,withonefieldperline.
Itisntpossibletosetmorethanonevariableintheinitializationpartwithoutusingamultipleassignmentstatement
suchasx=y=0.Thismakessenseonlyifalltheinitialvaluesareequal.(Butitispossibletoinitializeadditional
variablesbywritingtheirassignmentsasseparatestatementsprecedingtheforloop.)
Thesameistrueoftheincrementpart.Incrementingadditionalvariablesrequiresseparatestatementsattheendofthe
loop.TheCcompoundexpression,usingCscommaoperator,isusefulinthiscontext,butitisnotsupportedinawk.
Mostoften,incrementisanincrementexpression,asinthepreviousexample.Butthisisnotrequireditcanbeany
expressionwhatsoever.Forexample,thefollowingstatementprintsallthepowersoftwobetween1and100:
for(i=1;i<=100;i*=2)
printi
Ifthereisnothingtobedone,anyofthethreeexpressionsintheparenthesesfollowingtheforkeywordmaybe
omitted.Thus,for(;x>0;)isequivalenttowhile(x>0).Iftheconditionisomitted,itistreatedastrue,
effectivelyyieldinganinfiniteloop(i.e.,aloopthatneverterminates).
Inmostcases,aforloopisanabbreviationforawhileloop,asshownhere:
initialization
while(condition){
body
increment
}
Theonlyexceptioniswhenthecontinuestatement(seeContinueStatement)isusedinsidetheloop.Changingafor
statementtoawhilestatementinthiswaycanchangetheeffectofthecontinuestatementinsidetheloop.
Theawklanguagehasaforstatementinadditiontoawhilestatementbecauseaforloopisoftenbothlessworktotype
andmorenaturaltothinkof.Countingthenumberofiterationsisverycommoninloops.Itcanbeeasiertothinkofthis
countingaspartofloopingratherthanassomethingtodoinsidetheloop.
Thereisanalternativeversionoftheforloop,foriteratingoveralltheindicesofanarray:
for(iinarray)
dosomethingwitharray[i]
SeeScanninganArray,formoreinformationonthisversionoftheforloop.
Next:BreakStatement,Previous:ForStatement,Up:Statements[Contents][Index]
7.4.5TheswitchStatement
Thissectiondescribesagawkspecificfeature.Ifgawkisincompatibilitymode(seeOptions),itisnotavailable.
Theswitchstatementallowstheevaluationofanexpressionandtheexecutionofstatementsbasedonacasematch.
Casestatementsarecheckedforamatchintheordertheyaredefined.Ifnosuitablecaseisfound,thedefaultsectionis
executed,ifsupplied.
133/479
29/7/2016
TheGNUAwkUsersGuide
Eachcasecontainsasingleconstant,beitnumeric,string,orregexp.Theswitchexpressionisevaluated,andtheneach
casesconstantiscomparedagainsttheresultinturn.Thetypeofconstantdeterminesthecomparison:numericorstring
dotheusualcomparisons.Aregexpconstantdoesaregularexpressionmatchagainstthestringvalueoftheoriginal
expression.Thegeneralformoftheswitchstatementlookslikethis:
switch(expression){
casevalueorregularexpression:
casebody
default:
defaultbody
}
ControlflowintheswitchstatementworksasitdoesinC.Onceamatchtoagivencaseismade,thecasestatement
bodiesexecuteuntilabreak,continue,next,nextfile,orexitisencountered,ortheendoftheswitchstatementitself.
Forexample:
while((c=getopt(ARGC,ARGV,"aksx"))!=1){
switch(c){
case"a":
#reportsizeofallfiles
all_files=TRUE;
break
case"k":
BLOCK_SIZE=1024#1Kblocksize
break
case"s":
#dosumsonly
sum_only=TRUE
break
case"x":
#don'tcrossfilesystems
fts_flags=or(fts_flags,FTS_XDEV)
break
case"?":
default:
usage()
break
}
}
Notethatifnoneofthestatementsspecifiedherehaltexecutionofamatchedcasestatement,executionfallsthroughto
thenextcaseuntilexecutionhalts.Inthisexample,thecasefor"?"fallsthroughtothedefaultcase,whichistocalla
functionnamedusage().(Thegetopt()functionbeingcalledhereisdescribedinGetoptFunction.)
Next:ContinueStatement,Previous:SwitchStatement,Up:Statements[Contents][Index]
7.4.6ThebreakStatement
Thebreakstatementjumpsoutoftheinnermostfor,while,ordoloopthatenclosesit.Thefollowingexamplefindsthe
smallestdivisorofanyinteger,andalsoidentifiesprimenumbers:
#findsmallestdivisorofnum
{
num=$1
for(divisor=2;divisor*divisor<=num;divisor++){
if(num%divisor==0)
break
}
if(num%divisor==0)
printf"Smallestdivisorof%dis%d\n",num,divisor
else
printf"%disprime\n",num
}
134/479
29/7/2016
TheGNUAwkUsersGuide
Whentheremainderiszerointhefirstifstatement,awkimmediatelybreaksoutofthecontainingforloop.Thismeans
thatawkproceedsimmediatelytothestatementfollowingtheloopandcontinuesprocessing.(Thisisverydifferentfrom
theexitstatement,whichstopstheentireawkprogram.SeeExitStatement.)
Thefollowingprogramillustrateshowtheconditionofafororwhilestatementcouldbereplacedwithabreakinside
anif:
#findsmallestdivisorofnum
{
num=$1
for(divisor=2;;divisor++){
if(num%divisor==0){
printf"Smallestdivisorof%dis%d\n",num,divisor
break
}
if(divisor*divisor>num){
printf"%disprime\n",num
break
}
}
}
Thebreakstatementisalsousedtobreakoutoftheswitchstatement.ThisisdiscussedinSwitchStatement.
Thebreakstatementhasnomeaningwhenusedoutsidethebodyofalooporswitch.However,althoughitwasnever
documented,historicalimplementationsofawktreatedthebreakstatementoutsideofaloopasifitwereanext
statement(seeNextStatement).(d.c.)RecentversionsofBWKawknolongerallowthisusage,nordoesgawk.
Next:NextStatement,Previous:BreakStatement,Up:Statements[Contents][Index]
7.4.7ThecontinueStatement
Similartobreak,thecontinuestatementisusedonlyinsidefor,while,anddoloops.Itskipsovertherestoftheloop
body,causingthenextcyclearoundthelooptobeginimmediately.Contrastthiswithbreak,whichjumpsoutofthe
loopaltogether.
Thecontinuestatementinaforloopdirectsawktoskiptherestofthebodyoftheloopandresumeexecutionwiththe
incrementexpressionoftheforstatement.Thefollowingprogramillustratesthisfact:
BEGIN{
for(x=0;x<=20;x++){
if(x==5)
continue
printf"%d",x
}
print""
}
Thisprogramprintsallthenumbersfrom0to20exceptfor5,forwhichtheprintfisskipped.Becausetheincrement
x++isnotskipped, xdoesnotremainstuckat5.Contrasttheforloopfromthepreviousexamplewiththefollowing
whileloop:
BEGIN{
x=0
while(x<=20){
if(x==5)
continue
printf"%d",x
x++
}
print""
}
135/479
29/7/2016
TheGNUAwkUsersGuide
Thisprogramloopsforeveroncexreaches5,becausetheincrement(x++)isneverreached.
Thecontinuestatementhasnospecialmeaningwithrespecttotheswitchstatement,nordoesithaveanymeaning
whenusedoutsidethebodyofaloop.Historicalversionsofawktreatedacontinuestatementoutsidealoopthesame
waytheytreatedabreakstatementoutsidealoop:asifitwereanextstatement(seeNextStatement).(d.c.)Recent
versionsofBWKawknolongerworkthisway,nordoesgawk.
Next:NextfileStatement,Previous:ContinueStatement,Up:Statements[Contents][Index]
7.4.8ThenextStatement
Thenextstatementforcesawktoimmediatelystopprocessingthecurrentrecordandgoontothenextrecord.This
meansthatnofurtherrulesareexecutedforthecurrentrecord,andtherestofthecurrentrulesactionisntexecuted.
Contrastthiswiththeeffectofthegetlinefunction(seeGetline).Thatalsocausesawktoreadthenextrecord
immediately,butitdoesnotaltertheflowofcontrolinanyway(i.e.,therestofthecurrentactionexecuteswithanew
inputrecord).
Atthehighestlevel,awkprogramexecutionisaloopthatreadsaninputrecordandthentestseachrulespatternagainst
it.Ifyouthinkofthisloopasaforstatementwhosebodycontainstherules,thenthenextstatementisanalogoustoa
continuestatement.Itskipstotheendofthebodyofthisimplicitloopandexecutestheincrement(whichreadsanother
record).
Forexample,supposeanawkprogramworksonlyonrecordswithfourfields,anditshouldntfailwhengivenbadinput.
Toavoidcomplicatingtherestoftheprogram,writeaweedoutrulenearthebeginning,inthefollowingmanner:
NF!=4{
printf("%s:%d:skipped:NF!=4\n",FILENAME,FNR)>"/dev/stderr"
next
}
Becauseofthenextstatement,theprogramssubsequentruleswontseethebadrecord.Theerrormessageisredirected
tothestandarderroroutputstream,aserrormessagesshouldbe.Formoredetail,seeSpecialFiles.
Ifthenextstatementcausestheendoftheinputtobereached,thenthecodeinanyENDrulesisexecuted.See
BEGIN/END.
ThenextstatementisnotallowedinsideBEGINFILEandENDFILErules.SeeBEGINFILE/ENDFILE.
AccordingtothePOSIXstandard,thebehaviorisundefinedifthenextstatementisusedinaBEGINorENDrule.gawk
treatsitasasyntaxerror.AlthoughPOSIXdoesnotdisallowit,mostotherawkimplementationsdontallowthenext
statementinsidefunctionbodies(seeUserdefined).Justaswithanyothernextstatement,anextstatementinsidea
functionbodyreadsthenextrecordandstartsprocessingitwiththefirstruleintheprogram.
Next:ExitStatement,Previous:NextStatement,Up:Statements[Contents][Index]
7.4.9ThenextfileStatement
Thenextfilestatementissimilartothenextstatement.However,insteadofabandoningprocessingofthecurrent
record,thenextfilestatementinstructsawktostopprocessingthecurrentdatafile.
Uponexecutionofthenextfilestatement,FILENAMEisupdatedtothenameofthenextdatafilelistedonthecommand
line,FNRisresettoone,andprocessingstartsoverwiththefirstruleintheprogram.Ifthenextfilestatementcausesthe
endoftheinputtobereached,thenthecodeinanyENDrulesisexecuted.Anexceptiontothisiswhennextfileis
invokedduringexecutionofanystatementinanENDruleinthiscase,itcausestheprogramtostopimmediately.See
BEGIN/END.
136/479
29/7/2016
TheGNUAwkUsersGuide
Thenextfilestatementisusefulwhentherearemanydatafilestoprocessbutitisntnecessarytoprocesseveryrecord
ineveryfile.Withoutnextfile,inordertomoveontothenextdatafile,aprogramwouldhavetocontinuescanningthe
unwantedrecords.Thenextfilestatementaccomplishesthismuchmoreefficiently.
Ingawk,executionofnextfilecausesadditionalthingstohappen:anyENDFILErulesareexecutedifgawkisnotcurrently
inanENDorBEGINFILErule,ARGINDisincremented,andanyBEGINFILErulesareexecuted.(ARGINDhasntbeenintroduced
yet.SeeBuiltinVariables.)
Withgawk,nextfileisusefulinsideaBEGINFILEruletoskipoverafilethatwouldotherwisecausegawktoexitwitha
fatalerror.Inthiscase,ENDFILErulesarenotexecuted.SeeBEGINFILE/ENDFILE.
Althoughitmightseemthatclose(FILENAME)wouldaccomplishthesameasnextfile,thisisnttrue.close()is
reservedforclosingfiles,pipes,andcoprocessesthatareopenedwithredirections.Itisnotrelatedtothemain
processingthatawkdoeswiththefileslistedinARGV.
NOTE:Formanyyears,nextfilewasacommonextension.InSeptember2012,itwasacceptedfor
inclusionintothePOSIXstandard.SeetheAustinGroupwebsite.
ThecurrentversionofBWKawkandmawkalsosupportnextfile.However,theydontallowthenextfilestatement
insidefunctionbodies(seeUserdefined).gawkdoesanextfileinsideafunctionbodyreadsthenextrecordandstarts
processingitwiththefirstruleintheprogram,justasanyothernextfilestatement.
Previous:NextfileStatement,Up:Statements[Contents][Index]
7.4.10TheexitStatement
Theexitstatementcausesawktoimmediatelystopexecutingthecurrentruleandtostopprocessinginputany
remaininginputisignored.Theexitstatementiswrittenasfollows:
exit[returncode]
WhenanexitstatementisexecutedfromaBEGINrule,theprogramstopsprocessingeverythingimmediately.Noinput
recordsareread.However,ifanENDruleispresent,aspartofexecutingtheexitstatement,theENDruleisexecuted(see
BEGIN/END).IfexitisusedinthebodyofanENDrule,itcausestheprogramtostopimmediately.
AnexitstatementthatisnotpartofaBEGINorENDrulestopstheexecutionofanyfurtherautomaticrulesforthecurrent
record,skipsreadinganyremaininginputrecords,andexecutestheENDruleifthereisone.gawkalsoskipsanyENDFILE
rulestheydonotexecute.
Insuchacase,ifyoudontwanttheENDruletodoitsjob,setavariabletoanonzerovaluebeforetheexitstatement
andcheckthatvariableintheENDrule.SeeAssertFunction,foranexamplethatdoesthis.
Ifanargumentissuppliedtoexit,itsvalueisusedastheexitstatuscodefortheawkprocess.Ifnoargumentis
supplied,exitcausesawktoreturnasuccessstatus.Inthecasewhereanargumentissuppliedtoafirstexit
statement,andthenexitiscalledasecondtimefromanENDrulewithnoargument,awkusesthepreviouslysupplied
exitvalue.(d.c.)SeeExitStatus,formoreinformation.
Forexample,supposeanerrorconditionoccursthatisdifficultorimpossibletohandle.Conventionally,programs
reportthisbyexitingwithanonzerostatus.Anawkprogramcandothisusinganexitstatementwithanonzero
argument,asshowninthefollowingexample:
BEGIN{
if(("date"|getlinedate_now)<=0){
print"Can'tgetsystemdate">"/dev/stderr"
exit1
}
print"currentdateis",date_now
close("date")
}
137/479
29/7/2016
TheGNUAwkUsersGuide
NOTE:Forfullportability,exitvaluesshouldbebetweenzeroand126,inclusive.Negativevalues,and
valuesof127orgreater,maynotproduceconsistentresultsacrossdifferentoperatingsystems.
Next:PatternActionSummary,Previous:Statements,Up:PatternsandActions[Contents][Index]
7.5PredefinedVariables
Mostawkvariablesareavailabletouseforyourownpurposestheyneverchangeunlessyourprogramassignsvaluesto
them,andtheyneveraffectanythingunlessyourprogramexaminesthem.However,afewvariablesinawkhavespecial
builtinmeanings.awkexaminessomeoftheseautomatically,sothattheyenableyoutotellawkhowtodocertain
things.Othersaresetautomaticallybyawk,sothattheycarryinformationfromtheinternalworkingsofawktoyour
program.
Thissectiondocumentsallofgawkspredefinedvariables,mostofwhicharealsodocumentedinthechaptersdescribing
theirareasofactivity.
Usermodified:
Builtinvariablesthatyouchangetocontrolawk.
Autoset:
Builtinvariableswhereawkgivesyouinformation.
ARGCandARGV: WaystouseARGCandARGV.
Next:Autoset,Up:BuiltinVariables[Contents][Index]
7.5.1BuiltinVariablesThatControlawk
Thefollowingisanalphabeticallistofvariablesthatyoucanchangetocontrolhowawkdoescertainthings.
Thevariablesthatarespecifictogawkaremarkedwithapoundsign(#).Thesevariablesaregawkextensions.Inother
awkimplementationsorifgawkisincompatibilitymode(seeOptions),theyarenotspecial.(Anyexceptionsarenotedin
thedescriptionofeachvariable.)
BINMODE#
OnnonPOSIXsystems,thisvariablespecifiesuseofbinarymodeforallI/O.Numericvaluesofone,two,or
threespecifythatinputfiles,outputfiles,orallfiles,respectively,shouldusebinaryI/O.Anumericvalueless
thanzeroistreatedaszero,andanumericvaluegreaterthanthreeistreatedasthree.Alternatively,stringvalues
of"r"or"w"specifythatinputfilesandoutputfiles,respectively,shouldusebinaryI/O.Astringvalueof"rw"or
"wr"indicatesthatallfilesshouldusebinaryI/O.Anyotherstringvalueistreatedthesameas"rw",butcauses
gawktogenerateawarningmessage.BINMODEisdescribedinmoredetailinPCUsing.mawk(seeOtherVersions)
alsosupportsthisvariable,butonlyusingnumericvalues.
CONVFMT
Astringthatcontrolstheconversionofnumberstostrings(seeConversion).Itworksbybeingpassed,ineffect,
asthefirstargumenttothesprintf()function(seeStringFunctions).Itsdefaultvalueis"%.6g".CONVFMTwas
introducedbythePOSIXstandard.
FIELDWIDTHS#
Aspaceseparatedlistofcolumnsthattellsgawkhowtosplitinputwithfixedcolumnarboundaries.Assigninga
valuetoFIELDWIDTHSoverridestheuseofFSandFPATforfieldsplitting.SeeConstantSize,formoreinformation.
FPAT#
Aregularexpression(asastring)thattellsgawktocreatethefieldsbasedontextthatmatchestheregular
expression.AssigningavaluetoFPAToverridestheuseofFSandFIELDWIDTHSforfieldsplitting.SeeSplittingBy
Content,formoreinformation.
138/479
29/7/2016
TheGNUAwkUsersGuide
FS
Theinputfieldseparator(seeFieldSeparators).Thevalueisasinglecharacterstringoramulticharacterregular
expressionthatmatchestheseparationsbetweenfieldsinaninputrecord.Ifthevalueisthenullstring(""),then
eachcharacterintherecordbecomesaseparatefield.(Thisbehaviorisagawkextension.POSIXawkdoesnot
specifythebehaviorwhenFSisthenullstring.Nonetheless,someotherversionsofawkalsotreat""specially.)
Thedefaultvalueis"",astringconsistingofasinglespace.Asaspecialexception,thisvaluemeansthatany
sequenceofspaces,TABs,and/ornewlinesisasingleseparator.37Italsocausesspaces,TABs,andnewlinesat
thebeginningandendofarecordtobeignored.
YoucansetthevalueofFSonthecommandlineusingtheFoption:
awkF,'program'inputfiles
IfgawkisusingFIELDWIDTHSorFPATforfieldsplitting,assigningavaluetoFScausesgawktoreturntothenormal,
FSbasedfieldsplitting.AneasywaytodothisistosimplysayFS=FS,perhapswithanexplanatorycomment.
IGNORECASE#
IfIGNORECASEisnonzeroornonnull,thenallstringcomparisonsandallregularexpressionmatchingarecase
independent.Thisappliestoregexpmatchingwith~and!~,thegensub(),gsub(),index(),match(),
patsplit(),split(),andsub()functions,recordterminationwithRS,andfieldsplittingwithFSandFPAT.
However,thevalueofIGNORECASEdoesnotaffectarraysubscriptinganditdoesnotaffectfieldsplittingwhen
usingasinglecharacterfieldseparator.SeeCasesensitivity.
LINT#
Whenthisvariableistrue(nonzeroornonnull),gawkbehavesasifthelintcommandlineoptionisineffect
(seeOptions).Withavalueof"fatal",lintwarningsbecomefatalerrors.Withavalueof"invalid",only
warningsaboutthingsthatareactuallyinvalidareissued.(Thisisnotfullyimplementedyet.)Anyothertrue
valueprintsnonfatalwarnings.AssigningafalsevaluetoLINTturnsoffthelintwarnings.
Thisvariableisagawkextension.Itisnotspecialinotherawkimplementations.Unlikewiththeotherspecial
variables,changingLINTdoesaffecttheproductionoflintwarnings,evenifgawkisincompatibilitymode.Much
asthelintandtraditionaloptionsindependentlycontroldifferentaspectsofgawksbehavior,thecontrolof
lintwarningsduringprogramexecutionisindependentoftheflavorofawkbeingexecuted.
OFMT
Astringthatcontrolsconversionofnumberstostrings(seeConversion)forprintingwiththeprintstatement.It
worksbybeingpassedasthefirstargumenttothesprintf()function(seeStringFunctions).Itsdefaultvalueis
"%.6g".EarlierversionsofawkusedOFMTtospecifytheformatforconvertingnumberstostringsingeneral
expressionsthisisnowdonebyCONVFMT.
OFS
Theoutputfieldseparator(seeOutputSeparators).Itisoutputbetweenthefieldsprintedbyaprintstatement.Its
defaultvalueis"",astringconsistingofasinglespace.
ORS
Theoutputrecordseparator.Itisoutputattheendofeveryprintstatement.Itsdefaultvalueis"\n",thenewline
character.(SeeOutputSeparators.)
PREC#
Theworkingprecisionofarbitraryprecisionfloatingpointnumbers,53bitsbydefault(seeSettingprecision).
ROUNDMODE#
139/479
29/7/2016
TheGNUAwkUsersGuide
Theroundingmodetouseforarbitraryprecisionarithmeticonnumbers,bydefault"N"(roundTiesToEveninthe
IEEE754standardseeSettingtheroundingmode).
RS
Theinputrecordseparator.Itsdefaultvalueisastringcontainingasinglenewlinecharacter,whichmeansthatan
inputrecordconsistsofasinglelineoftext.Itcanalsobethenullstring,inwhichcaserecordsareseparatedby
runsofblanklines.Ifitisaregexp,recordsareseparatedbymatchesoftheregexpintheinputtext.(See
Records.)
TheabilityforRStobearegularexpressionisagawkextension.Inmostotherawkimplementations,orifgawkisin
compatibilitymode(seeOptions),justthefirstcharacterofRSsvalueisused.
SUBSEP
Thesubscriptseparator.Ithasthedefaultvalueof"\034"andisusedtoseparatethepartsoftheindicesofa
multidimensionalarray.Thus,theexpressionfoo["A","B"]reallyaccesses foo["A\034B"](see
Multidimensional).
TEXTDOMAIN#
Usedforinternationalizationofprogramsattheawklevel.Itsetsthedefaulttextdomainforspeciallymarked
stringconstantsinthesourcetext,aswellasforthedcgettext(),dcngettext(),andbindtextdomain()functions
(seeInternationalization).ThedefaultvalueofTEXTDOMAINis"messages".
Next:ARGCandARGV,Previous:Usermodified,Up:BuiltinVariables[Contents][Index]
7.5.2BuiltinVariablesThatConveyInformation
Thefollowingisanalphabeticallistofvariablesthatawksetsautomaticallyoncertainoccasionsinordertoprovide
informationtoyourprogram.
Thevariablesthatarespecifictogawkaremarkedwithapoundsign(#).Thesevariablesaregawkextensions.Inother
awkimplementationsorifgawkisincompatibilitymode(seeOptions),theyarenotspecial:
ARGC,ARGV
ThecommandlineargumentsavailabletoawkprogramsarestoredinanarraycalledARGV.ARGCisthenumberof
commandlineargumentspresent.SeeOtherArguments.Unlikemostawkarrays,ARGVisindexedfrom0toARGC
1.Inthefollowingexample:
$awk'BEGIN{
>for(i=0;i<ARGC;i++)
>printARGV[i]
>}'inventoryshippedmaillist
|awk
|inventoryshipped
|maillist
ARGV[0]containsawk,ARGV[1]containsinventoryshipped,andARGV[2]containsmaillist.Thevalueof
ARGCisthree,onemorethantheindexofthelastelementinARGV,becausetheelementsarenumberedfromzero.
ThenamesARGCandARGV,aswellastheconventionofindexingthearrayfrom0toARGC1,arederivedfromthe
Clanguagesmethodofaccessingcommandlinearguments.
ThevalueofARGV[0]canvaryfromsystemtosystem.Also,youshouldnotethattheprogramtextisnotincluded
inARGV,norareanyofawkscommandlineoptions.SeeARGCandARGV,forinformationabouthowawkuses
thesevariables.(d.c.)
ARGIND#
140/479
29/7/2016
TheGNUAwkUsersGuide
TheindexinARGVofthecurrentfilebeingprocessed.Everytimegawkopensanewdatafileforprocessing,itsets
ARGINDtotheindexinARGVofthefilename.Whengawkisprocessingtheinputfiles,FILENAME==ARGV[ARGIND]
isalwaystrue.
Thisvariableisusefulinfileprocessingitallowsyoutotellhowfaralongyouareinthelistofdatafilesaswell
astodistinguishbetweensuccessiveinstancesofthesamefilenameonthecommandline.
WhileyoucanchangethevalueofARGINDwithinyourawkprogram,gawkautomaticallysetsittoanewvalue
whenitopensthenextfile.
ENVIRON
Anassociativearraycontainingthevaluesoftheenvironment.Thearrayindicesaretheenvironmentvariable
namestheelementsarethevaluesoftheparticularenvironmentvariables.Forexample,ENVIRON["HOME"]might
be"/home/arnold".Changingthisarraydoesnotaffecttheenvironmentpassedontoanyprogramsthatawkmay
spawnviaredirectionorthesystem()function.(Inafutureversionofgawk,itmaydoso.)
Someoperatingsystemsmaynothaveenvironmentvariables.Onsuchsystems,theENVIRONarrayisempty
(exceptforENVIRON["AWKPATH"]andENVIRON["AWKLIBPATH"]seeAWKPATHVariable,andseeAWKLIBPATH
Variable).
ERRNO#
Ifasystemerroroccursduringaredirectionforgetline,duringareadforgetline,orduringaclose()operation,
thenERRNOcontainsastringdescribingtheerror.
Inaddition,gawkclearsERRNObeforeopeningeachcommandlineinputfile.Thisenablescheckingifthefileis
readableinsideaBEGINFILEpattern(seeBEGINFILE/ENDFILE).
Otherwise,ERRNOworkssimilarlytotheCvariableerrno.Exceptforthecasejustmentioned,gawkneverclearsit
(setsittozeroor"").Thus,youshouldonlyexpectitsvaluetobemeaningfulwhenanI/Ooperationreturnsa
failurevalue,suchasgetlinereturning1.Youare,ofcourse,freetoclearityourselfbeforedoinganI/O
operation.
FILENAME
Thenameofthecurrentinputfile.Whennodatafilesarelistedonthecommandline,awkreadsfromthestandard
inputandFILENAMEissetto"".FILENAMEchangeseachtimeanewfileisread(seeReadingFiles).InsideaBEGIN
rule,thevalueofFILENAMEis"",becausetherearenoinputfilesbeingprocessedyet.38(d.c.)Note,though,that
usinggetline(seeGetline)insideaBEGINrulecangiveFILENAMEavalue.
FNR
Thecurrentrecordnumberinthecurrentfile.awkincrementsFNReachtimeitreadsanewrecord(seeRecords).
awkresetsFNRtozeroeachtimeitstartsanewinputfile.
NF
Thenumberoffieldsinthecurrentinputrecord.NFisseteachtimeanewrecordisread,whenanewfieldis
created,orwhen$0changes(seeFields).
Unlikemostofthevariablesdescribedinthissubsection,assigningavaluetoNFhasthepotentialtoaffectawks
internalworkings.Inparticular,assignmentstoNFcanbeusedtocreatefieldsinorremovefieldsfromthecurrent
record.SeeChangingFields.
FUNCTAB#
Anarraywhoseindicesandcorrespondingvaluesarethenamesofallthebuiltin,userdefined,andextension
functionsintheprogram.
141/479
29/7/2016
TheGNUAwkUsersGuide
NOTE:AttemptingtousethedeletestatementwiththeFUNCTABarraycausesafatalerror.Any
attempttoassigntoanelementofFUNCTABalsocausesafatalerror.
NR
Thenumberofinputrecordsawkhasprocessedsincethebeginningoftheprogramsexecution(seeRecords).awk
incrementsNReachtimeitreadsanewrecord.
PROCINFO#
Theelementsofthisarrayprovideaccesstoinformationabouttherunningawkprogram.Thefollowingelements
(listedalphabetically)areguaranteedtobeavailable:
PROCINFO["egid"]
Thevalueofthegetegid()systemcall.
PROCINFO["euid"]
Thevalueofthegeteuid()systemcall.
PROCINFO["FS"]
Thisis"FS"iffieldsplittingwithFSisineffect,"FIELDWIDTHS"iffieldsplittingwithFIELDWIDTHSisin
effect,or"FPAT"iffieldmatchingwithFPATisineffect.
PROCINFO["identifiers"]
Asubarray,indexedbythenamesofallidentifiersusedinthetextoftheawkprogram.Anidentifieris
simplythenameofavariable(beitscalarorarray),builtinfunction,userdefinedfunction,orextension
function.Foreachidentifier,thevalueoftheelementisoneofthefollowing:
"array"
Theidentifierisanarray.
"builtin"
Theidentifierisabuiltinfunction.
"extension"
Theidentifierisanextensionfunctionloadedvia@loadorl.
"scalar"
Theidentifierisascalar.
"untyped"
Theidentifierisuntyped(couldbeusedasascalaroranarraygawkdoesntknowyet).
"user"
Theidentifierisauserdefinedfunction.
Thevaluesindicatewhatgawkknowsabouttheidentifiersafterithasfinishedparsingtheprogramtheyare
notupdatedwhiletheprogramruns.
PROCINFO["gid"]
Thevalueofthegetgid()systemcall.
142/479
29/7/2016
TheGNUAwkUsersGuide
PROCINFO["pgrpid"]
TheprocessgroupIDofthecurrentprocess.
PROCINFO["pid"]
TheprocessIDofthecurrentprocess.
PROCINFO["ppid"]
TheparentprocessIDofthecurrentprocess.
PROCINFO["sorted_in"]
IfthiselementexistsinPROCINFO,itsvaluecontrolstheorderinwhicharrayindiceswillbeprocessedby
for(indxinarray)loops.Thisisanadvancedfeature,sowedeferthefulldescriptionuntillatersee
ScanninganArray.
PROCINFO["strftime"]
Thedefaulttimeformatstringforstrftime().Assigninganewvaluetothiselementchangesthedefault.
SeeTimeFunctions.
PROCINFO["uid"]
Thevalueofthegetuid()systemcall.
PROCINFO["version"]
Theversionofgawk.
ThefollowingadditionalelementsinthearrayareavailabletoprovideinformationabouttheMPFRandGMP
librariesifyourversionofgawksupportsarbitraryprecisionarithmetic(seeArbitraryPrecisionArithmetic):
PROCINFO["mpfr_version"]
TheversionoftheGNUMPFRlibrary.
PROCINFO["gmp_version"]
TheversionoftheGNUMPlibrary.
PROCINFO["prec_max"]
ThemaximumprecisionsupportedbyMPFR.
PROCINFO["prec_min"]
TheminimumprecisionrequiredbyMPFR.
Thefollowingadditionalelementsinthearrayareavailabletoprovideinformationabouttheversionofthe
extensionAPI,ifyourversionofgawksupportsdynamicloadingofextensionfunctions(seeDynamic
Extensions):
PROCINFO["api_major"]
ThemajorversionoftheextensionAPI.
PROCINFO["api_minor"]
TheminorversionoftheextensionAPI.
Onsomesystems,theremaybeelementsinthearray,"group1"through"groupN"forsomeN.Nisthenumberof
143/479
29/7/2016
TheGNUAwkUsersGuide
supplementarygroupsthattheprocesshas.Usetheinoperatortotestfortheseelements(seeReferenceto
Elements).
ThePROCINFOarrayhasthefollowingadditionaluses:
Itmaybeusedtoprovideatimeoutwhenreadingfromanyopeninputfile,pipe,orcoprocess.SeeRead
Timeout,formoreinformation.
Itmaybeusedtocausecoprocessestocommunicateoverpseudottysinsteadofthroughtwowaypipes
thisisdiscussedfurtherinTwowayI/O.
RLENGTH
Thelengthofthesubstringmatchedbythematch()function(seeStringFunctions).RLENGTHissetbyinvokingthe
match()function.Itsvalueisthelengthofthematchedstring,or1ifnomatchisfound.
RSTART
Thestartindexincharactersofthesubstringthatismatchedbythematch()function(seeStringFunctions).
RSTARTissetbyinvokingthematch()function.Itsvalueisthepositionofthestringwherethematchedsubstring
starts,orzeroifnomatchwasfound.
RT#
TheinputtextthatmatchedthetextdenotedbyRS,therecordseparator.Itisseteverytimearecordisread.
SYMTAB#
Anarraywhoseindicesarethenamesofalldefinedglobalvariablesandarraysintheprogram.SYMTABmakes
gawkssymboltablevisibletotheawkprogrammer.Itisbuiltasgawkparsestheprogramandiscompletebefore
theprogramstartstorun.
Thearraymaybeusedforindirectaccesstoreadorwritethevalueofavariable:
foo=5
SYMTAB["foo"]=4
printfoo#prints4
Theisarray()function(seeTypeFunctions)maybeusedtotestifanelementinSYMTABisanarray.Also,you
maynotusethedeletestatementwiththeSYMTABarray.
YoumayuseanindexforSYMTABthatisnotapredefinedidentifier:
SYMTAB["xxx"]=5
printSYMTAB["xxx"]
Thisworksasexpected:inthiscaseSYMTABactsjustlikearegulararray.Theonlydifferenceisthatyoucantthen
deleteSYMTAB["xxx"].
TheSYMTABarrayismoreinterestingthanitlooks.AndrewSchorrpointsoutthatiteffectivelygivesawkdata
pointers.Considerhisexample:
#Indirectmultiplyofanyvariablebyamount,returnresult
functionmultiply(variable,amount)
{
returnSYMTAB[variable]*=amount
}
NOTE:Inordertoavoidseveretimetravelparadoxes,39neitherFUNCTABnorSYMTABisavailableas
anelementwithintheSYMTABarray.
ChangingNRandFNR
144/479
29/7/2016
TheGNUAwkUsersGuide
awkincrementsNRandFNReachtimeitreadsarecord,insteadofsettingthemtotheabsolutevalueofthenumberof
recordsread.Thismeansthataprogramcanchangethesevariablesandtheirnewvaluesareincrementedforeach
record.(d.c.)Thefollowingexampleshowsthis:
$echo'1
>2
>3
>4'|awk'NR==2{NR=17}
>{printNR}'
|1
|17
|18
|19
BeforeFNRwasaddedtotheawklanguage(seeV7/SVR3.1),manyawkprogramsusedthisfeaturetotrackthenumber
ofrecordsinafilebyresettingNRtozerowhenFILENAMEchanged.
Previous:Autoset,Up:BuiltinVariables[Contents][Index]
7.5.3UsingARGCandARGV
Autoset,presentedthefollowingprogramdescribingtheinformationcontainedinARGCandARGV:
$awk'BEGIN{
>for(i=0;i<ARGC;i++)
>printARGV[i]
>}'inventoryshippedmaillist
|awk
|inventoryshipped
|maillist
Inthisexample,ARGV[0]containsawk,ARGV[1]containsinventoryshipped,andARGV[2]containsmaillist.
NoticethattheawkprogramisnotenteredinARGV.Theothercommandlineoptions,withtheirarguments,arealsonot
entered.Thisincludesvariableassignmentsdonewiththevoption(seeOptions).Normalvariableassignmentsonthe
commandlinearetreatedasargumentsanddoshowupintheARGVarray.Giventhefollowingprograminafilenamed
showargs.awk:
BEGIN{
printf"A=%d,B=%d\n",A,B
for(i=0;i<ARGC;i++)
printf"\tARGV[%d]=%s\n",i,ARGV[i]
}
END{printf"A=%d,B=%d\n",A,B}
Runningitproducesthefollowing:
$awkvA=1fshowargs.awkB=2/dev/null
|A=1,B=0
|ARGV[0]=awk
|ARGV[1]=B=2
|ARGV[2]=/dev/null
|A=1,B=2
AprogramcanalterARGCandtheelementsofARGV.Eachtimeawkreachestheendofaninputfile,itusesthenext
elementofARGVasthenameofthenextinputfile.Bystoringadifferentstringthere,aprogramcanchangewhichfiles
areread.Use""torepresentthestandardinput.StoringadditionalelementsandincrementingARGCcausesadditional
filestoberead.
IfthevalueofARGCisdecreased,thateliminatesinputfilesfromtheendofthelist.ByrecordingtheoldvalueofARGC
elsewhere,aprogramcantreattheeliminatedargumentsassomethingotherthanfilenames.
145/479
29/7/2016
TheGNUAwkUsersGuide
Toeliminateafilefromthemiddleofthelist,storethenullstring("")intoARGVinplaceofthefilesname.Asaspecial
feature,awkignoresfilenamesthathavebeenreplacedwiththenullstring.Anotheroptionistousethedeletestatement
toremoveelementsfromARGV(seeDelete).
AlloftheseactionsaretypicallydoneintheBEGINrule,beforeactualprocessingoftheinputbegins.SeeSplitProgram,
andseeTeeProgram,forexamplesofeachwayofremovingelementsfromARGV.
Toactuallygetoptionsintoanawkprogram,endtheawkoptionswithandthensupplytheawkprogramsoptions,in
thefollowingmanner:
awkfmyprog.awkvqfile1file2
ThefollowingfragmentprocessesARGVinordertoexamine,andthenremove,thepreviouslymentionedcommandline
options:
BEGIN{
for(i=1;i<ARGC;i++){
if(ARGV[i]=="v")
verbose=1
elseif(ARGV[i]=="q")
debug=1
elseif(ARGV[i]~/^./){
e=sprintf("%s:unrecognizedoption%c",
ARGV[0],substr(ARGV[i],2,1))
printe>"/dev/stderr"
}else
break
deleteARGV[i]
}
}
Endingtheawkoptionswithisntnecessaryingawk.Unlessposixhasbeenspecified,gawksilentlyputsany
unrecognizedoptionsintoARGVfortheawkprogramtodealwith.Assoonasitseesanunknownoption,gawkstops
lookingforotheroptionsthatitmightotherwiserecognize.Thepreviouscommandlinewithgawkwouldbe:
gawkfmyprog.awkqvfile1file2
Becauseqisnotavalidgawkoption,itandthefollowingvarepassedontotheawkprogram.(SeeGetoptFunction,
foranawklibraryfunctionthatparsescommandlineoptions.)
Whendesigningyourprogram,youshouldchooseoptionsthatdontconflictwithgawks,becauseitwillprocessany
optionsthatitacceptsbeforepassingtherestofthecommandlineontoyourprogram.Using#!withtheEoption
mayhelp(seeExecutableScripts,andseeOptions,).
Previous:BuiltinVariables,Up:PatternsandActions[Contents][Index]
7.6Summary
Patternactionpairsmakeupthebasicelementsofanawkprogram.Patternsareeithernormalexpressions,range
expressions,orregexpconstantsoneofthespecialkeywordsBEGIN,END,BEGINFILE,orENDFILEorempty.The
actionexecutesifthecurrentrecordmatchesthepattern.Empty(missing)patternsmatchallrecords.
I/OfromBEGINandENDruleshascertainconstraints.Thisisalsotrue,onlymoreso,forBEGINFILEandENDFILE
rules.Thelattertwogiveyouhooksintogawksfileprocessing,allowingyoutorecoverfromafilethat
otherwisewouldcauseafatalerror(suchasafilethatcannotbeopened).
Shellvariablescanbeusedinawkprogramsbycarefuluseofshellquoting.Itiseasiertopassashellvariableinto
awkbyusingthevoptionandanawkvariable.
Actionsconsistofstatementsenclosedincurlybraces.Statementsarebuiltupfromexpressions,control
statements,compoundstatements,inputandoutputstatements,anddeletionstatements.
Thecontrolstatementsinawkareifelse,while,for,anddowhile.gawkaddstheswitchstatement.Therearetwo
flavorsofforstatement:oneforperforminggenerallooping,andtheotherforiteratingthroughanarray.
146/479
29/7/2016
TheGNUAwkUsersGuide
breakandcontinueletyouexitearlyorstartthenextiterationofaloop(orgetoutofaswitch).
nextandnextfileletyoureadthenextrecordandstartoveratthetopofyourprogramorskiptothenextinput
fileandstartover,respectively.
Theexitstatementterminatesyourprogram.Whenexecutedfromanaction(orfunctionbody),ittransfers
controltotheENDstatements.FromanENDstatementbody,itexitsimmediately.Youmaypassanoptional
numericvaluetobeusedasawksexitstatus.
Somepredefinedvariablesprovidecontroloverawk,mainlyforI/O.Othervariablesconveyinformationfromawk
toyourprogram.
ARGCandARGVmakethecommandlineargumentsavailabletoyourprogram.ManipulatingthemfromaBEGINrule
letsyoucontrolhowawkwillprocesstheprovideddatafiles.
Next:Functions,Previous:PatternsandActions,Up:Top[Contents][Index]
8Arraysinawk
Anarrayisatableofvaluescalledelements.Theelementsofanarrayaredistinguishedbytheirindices.Indicesmaybe
eithernumbersorstrings.
Thischapterdescribeshowarraysworkinawk,howtousearrayelements,howtoscanthrougheveryelementinan
array,andhowtoremovearrayelements.Italsodescribeshowawksimulatesmultidimensionalarrays,aswellassome
ofthelessobviouspointsaboutarrayusage.Thechaptermovesontodiscussgawksfacilityforsortingarrays,andends
withabriefdescriptionofgawksabilitytosupporttruearraysofarrays.
ArrayBasics:
Thebasicsofarrays.
NumericArraySubscripts: Howtousenumbersassubscriptsinawk.
UninitializedSubscripts: UsingUninitializedvariablesassubscripts.
Delete:
Thedeletestatementremovesanelementfromanarray.
Multidimensional:
Emulatingmultidimensionalarraysinawk.
ArraysofArrays:
Truemultidimensionalarrays.
ArraysSummary:
Summaryofarrays.
Next:NumericArraySubscripts,Up:Arrays[Contents][Index]
8.1TheBasicsofArrays
Thissectionpresentsthebasics:workingwithelementsinarraysoneatatime,andtraversingalloftheelementsinan
array.
ArrayIntro:
IntroductiontoArrays
ReferencetoElements: Howtoexamineoneelementofanarray.
AssigningElements: Howtochangeanelementofanarray.
ArrayExample:
BasicExampleofanArray
ScanninganArray:
Avariationoftheforstatement.Itloopsthroughtheindicesofanarraysexistingelements.
ControllingScanning: Controllingtheorderinwhicharraysarescanned.
Next:ReferencetoElements,Up:ArrayBasics[Contents][Index]
8.1.1IntroductiontoArrays
DoinglinearscansoveranassociativearrayisliketryingtoclubsomeonetodeathwithaloadedUzi.
LarryWall
147/479
29/7/2016
TheGNUAwkUsersGuide
Theawklanguageprovidesonedimensionalarraysforstoringgroupsofrelatedstringsornumbers.Everyawkarray
musthaveaname.Arraynameshavethesamesyntaxasvariablenamesanyvalidvariablenamewouldalsobeavalid
arrayname.Butonenamecannotbeusedinbothways(asanarrayandasavariable)inthesameawkprogram.
Arraysinawksuperficiallyresemblearraysinotherprogramminglanguages,buttherearefundamentaldifferences.In
awk,itisntnecessarytospecifythesizeofanarraybeforestartingtouseit.Additionally,anynumberorstring,notjust
consecutiveintegers,maybeusedasanarrayindex.
Inmostotherlanguages,arraysmustbedeclaredbeforeuse,includingaspecificationofhowmanyelementsor
componentstheycontain.Insuchlanguages,thedeclarationcausesacontiguousblockofmemorytobeallocatedfor
thatmanyelements.Usually,anindexinthearraymustbeanonnegativeinteger.Forexample,theindexzerospecifies
thefirstelementinthearray,whichisactuallystoredatthebeginningoftheblockofmemory.Indexonespecifiesthe
secondelement,whichisstoredinmemoryrightafterthefirstelement,andsoon.Itisimpossibletoaddmoreelements
tothearray,becauseithasroomonlyforasmanyelementsasgiveninthedeclaration.(Somelanguagesallowarbitrary
startingandendingindicese.g.,15..27butthesizeofthearrayisstillfixedwhenthearrayisdeclared.)
AcontiguousarrayoffourelementsmightlooklikeFigure8.1,conceptually,iftheelementvaluesareeight,"foo","",
and30.
Figure8.1:Acontiguousarray
Onlythevaluesarestoredtheindicesareimplicitfromtheorderofthevalues.Here,eightisthevalueatindexzero,
becauseeightappearsinthepositionwithzeroelementsbeforeit.
Arraysinawkaredifferenttheyareassociative.Thismeansthateacharrayisacollectionofpairsanindexandits
correspondingarrayelementvalue:
Index
3
1
0
2
Value
30
"foo"
8
""
Thepairsareshowninjumbledorderbecausetheirorderisirrelevant.40
Oneadvantageofassociativearraysisthatnewpairscanbeaddedatanytime.Forexample,supposeatenthelementis
addedtothearraywhosevalueis"numberten".Theresultis:
Index
10
3
1
0
2
Value
"numberten"
30
"foo"
8
""
Nowthearrayissparse,whichjustmeanssomeindicesaremissing.Ithaselements03and10,butdoesnthave
elements4,5,6,7,8,or9.
Anotherconsequenceofassociativearraysisthattheindicesdonthavetobenonnegativeintegers.Anynumber,or
evenastring,canbeanindex.Forexample,thefollowingisanarraythattranslateswordsfromEnglishtoFrench:
Index
Value
148/479
29/7/2016
TheGNUAwkUsersGuide
"dog"
"cat"
"one"
1
"chien"
"chat"
"un"
"un"
Herewedecidedtotranslatethenumberoneinbothspelledoutandnumericformthusillustratingthatasinglearray
canhavebothnumbersandstringsasindices.(Infact,arraysubscriptsarealwaysstrings.Therearesomesubtletiesto
hownumbersworkwhenusedasarraysubscriptsthisisdiscussedinmoredetailinNumericArraySubscripts.)Here,
thenumber1isntdoublequoted,becauseawkautomaticallyconvertsittoastring.
ThevalueofIGNORECASEhasnoeffectuponarraysubscripting.Theidenticalstringvalueusedtostoreanarrayelement
mustbeusedtoretrieveit.Whenawkcreatesanarray(e.g.,withthesplit()builtinfunction),thatarraysindicesare
consecutiveintegersstartingatone.(SeeStringFunctions.)
awksarraysareefficientthetimetoaccessanelementisindependentofthenumberofelementsinthearray.
Next:AssigningElements,Previous:ArrayIntro,Up:ArrayBasics[Contents][Index]
8.1.2ReferringtoanArrayElement
Theprincipalwaytouseanarrayistorefertooneofitselements.Anarrayreferenceisanexpressionasfollows:
array[indexexpression]
Here,arrayisthenameofanarray.Theexpressionindexexpressionistheindexofthedesiredelementofthearray.
Thevalueofthearrayreferenceisthecurrentvalueofthatarrayelement.Forexample,foo[4.3]isanexpression
referencingtheelementofarrayfooatindex4.3.
Areferencetoanarrayelementthathasnorecordedvalueyieldsavalueof"",thenullstring.Thisincludeselements
thathavenotbeenassignedanyvalueaswellaselementsthathavebeendeleted(seeDelete).
NOTE:Areferencetoanelementthatdoesnotexistautomaticallycreatesthatarrayelement,withthenull
stringasitsvalue.(Insomecases,thisisunfortunate,becauseitmightwastememoryinsideawk.)
Noviceawkprogrammersoftenmakethemistakeofcheckingifanelementexistsbycheckingifthevalue
isempty:
#Checkif"foo"existsina:Incorrect!
if(a["foo"]!="")
Thisisincorrectfortworeasons.First,itcreatesa["foo"]ifitdidntexistbefore!Second,itisvalid(ifa
bitunusual)tosetanarrayelementequaltotheemptystring.
Todeterminewhetheranelementexistsinanarrayatacertainindex,usethefollowingexpression:
indxinarray
Thisexpressiontestswhethertheparticularindexindxexists,withoutthesideeffectofcreatingthatelementifitisnot
present.Theexpressionhasthevalueone(true)ifarray[indx]existsandzero(false)ifitdoesnotexist.(Weuseindx
here,becauseindexisthenameofabuiltinfunction.)Forexample,thisstatementtestswhetherthearrayfrequencies
containstheindex2:
if(2infrequencies)
print"Subscript2ispresent."
Notethatthisisnotatestofwhetherthearrayfrequenciescontainsanelementwhosevalueistwo.Thereisnowayto
dothatexcepttoscanalltheelements.Also,thisdoesnotcreatefrequencies[2],whilethefollowing(incorrect)
alternativedoes:
149/479
29/7/2016
TheGNUAwkUsersGuide
if(frequencies[2]!="")
print"Subscript2ispresent."
Next:ArrayExample,Previous:ReferencetoElements,Up:ArrayBasics[Contents][Index]
8.1.3AssigningArrayElements
Arrayelementscanbeassignedvaluesjustlikeawkvariables:
array[indexexpression]=value
arrayisthenameofanarray.Theexpressionindexexpressionistheindexoftheelementofthearraythatisassigneda
value.Theexpressionvalueisthevaluetoassigntothatelementofthearray.
Next:ScanninganArray,Previous:AssigningElements,Up:ArrayBasics[Contents][Index]
8.1.4BasicArrayExample
Thefollowingprogramtakesalistoflines,eachbeginningwithalinenumber,andprintsthemoutinorderofline
number.Thelinenumbersarenotinorderwhentheyarefirstreadinstead,theyarescrambled.Thisprogramsortsthe
linesbymakinganarrayusingthelinenumbersassubscripts.Theprogramthenprintsoutthelinesinsortedorderof
theirnumbers.Itisaverysimpleprogramandgetsconfuseduponencounteringrepeatednumbers,gaps,orlinesthat
dontbeginwithanumber:
{
if($1>max)
max=$1
arr[$1]=$0
}
END{
for(x=1;x<=max;x++)
printarr[x]
}
Thefirstrulekeepstrackofthelargestlinenumberseensofaritalsostoreseachlineintothearrayarr,atanindexthat
isthelinesnumber.Thesecondrulerunsafteralltheinputhasbeenread,toprintoutallthelines.Whenthisprogram
isrunwiththefollowinginput:
5IamtheFiveman
2Whoareyou?Thenewnumbertwo!
4...Andfouronthefloor
1Whoisnumberone?
3Ithreeyou.
Itsoutputis:
1Whoisnumberone?
2Whoareyou?Thenewnumbertwo!
3Ithreeyou.
4...Andfouronthefloor
5IamtheFiveman
Ifalinenumberisrepeated,thelastlinewithagivennumberoverridestheothers.Gapsinthelinenumberscanbe
handledwithaneasyimprovementtotheprogramsENDrule,asfollows:
END{
for(x=1;x<=max;x++)
if(xinarr)
printarr[x]
}
150/479
29/7/2016
TheGNUAwkUsersGuide
Next:ControllingScanning,Previous:ArrayExample,Up:ArrayBasics[Contents][Index]
8.1.5ScanningAllElementsofanArray
Inprogramsthatusearrays,itisoftennecessarytousealoopthatexecutesonceforeachelementofanarray.Inother
languages,wherearraysarecontiguousandindicesarelimitedtononnegativeintegers,thisiseasy:allthevalidindices
canbefoundbycountingfromthelowestindexuptothehighest.Thistechniquewontdothejobinawk,becauseany
numberorstringcanbeanarrayindex.Soawkhasaspecialkindofforstatementforscanninganarray:
for(varinarray)
body
Thisloopexecutesbodyonceforeachindexinarraythattheprogramhaspreviouslyused,withthevariablevarsetto
thatindex.
Thefollowingprogramusesthisformoftheforstatement.Thefirstrulescanstheinputrecordsandnoteswhichwords
appear(atleastonce)intheinput,bystoringaoneintothearrayusedwiththewordastheindex.Thesecondrulescans
theelementsofusedtofindallthedistinctwordsthatappearintheinput.Itprintseachwordthatismorethan10
characterslongandalsoprintsthenumberofsuchwords.SeeStringFunctions,formoreinformationonthebuiltin
functionlength().
#Recorda1foreachwordthatisusedatleastonce
{
for(i=1;i<=NF;i++)
used[$i]=1
}
#Findnumberofdistinctwordsmorethan10characterslong
END{
for(xinused){
if(length(x)>10){
++num_long_words
printx
}
}
printnum_long_words,"wordslongerthan10characters"
}
SeeWordSorting,foramoredetailedexampleofthistype.
Theorderinwhichelementsofthearrayareaccessedbythisstatementisdeterminedbytheinternalarrangementofthe
arrayelementswithinawkandinstandardawkcannotbecontrolledorchanged.Thiscanleadtoproblemsifnew
elementsareaddedtoarraybystatementsintheloopbodyitisnotpredictablewhethertheforloopwillreachthem.
Similarly,changingvarinsidetheloopmayproducestrangeresults.Itisbesttoavoidsuchthings.
Asapointofinformation,gawksetsupthelistofelementstobeiteratedoverbeforetheloopstarts,anddoesnotchange
it.Butnotallawkversionsdoso.Considerthisprogram,namedloopcheck.awk:
BEGIN{
a["here"]="here"
a["is"]="is"
a["a"]="a"
a["loop"]="loop"
for(iina){
j++
a[j]=j
printi
}
}
Hereiswhathappenswhenrunwithgawk(andmawk):
151/479
29/7/2016
TheGNUAwkUsersGuide
$gawkfloopcheck.awk
|here
|loop
|a
|is
ContrastthistoBWKawk:
$nawkfloopcheck.awk
|loop
|here
|is
|a
|1
Previous:ScanninganArray,Up:ArrayBasics[Contents][Index]
8.1.6UsingPredefinedArrayScanningOrderswithgawk
Thissubsectiondescribesafeaturethatisspecifictogawk.
Bydefault,whenaforlooptraversesanarray,theorderisundefined,meaningthattheawkimplementationdetermines
theorderinwhichthearrayistraversed.Thisorderisusuallybasedontheinternalimplementationofarraysandwill
varyfromoneversionofawktothenext.
Often,though,youmaywishtodosomethingsimple,suchastraversethearraybycomparingtheindicesinascending
order,ortraversethearraybycomparingthevaluesindescendingorder.gawkprovidestwomechanismsthatgive
youthiscontrol:
SetPROCINFO["sorted_in"]tooneofasetofpredefinedvalues.Wedescribethisnow.
SetPROCINFO["sorted_in"]tothenameofauserdefinedfunctiontouseforcomparisonofarrayelements.This
advancedfeatureisdescribedlaterinArraySorting.
ThefollowingspecialvaluesforPROCINFO["sorted_in"]areavailable:
"@unsorted"
Arrayelementsareprocessedinarbitraryorder,whichisthedefaultawkbehavior.
"@ind_str_asc"
Orderbyindicesinascendingordercomparedasstringsthisisthemostbasicsort.(Internally,arrayindicesare
alwaysstrings,sowitha[2*5]=1theindexis "10"ratherthannumeric10.)
"@ind_num_asc"
Orderbyindicesinascendingorderbutforcethemtobetreatedasnumbersintheprocess.Anyindexwithanon
numericvaluewillenduppositionedasifitwerezero.
"@val_type_asc"
Orderbyelementvaluesinascendingorder(ratherthanbyindices).Orderingisbythetypeassignedtothe
element(seeTypingandComparison).Allnumericvaluescomebeforeallstringvalues,whichinturncome
beforeallsubarrays.(SubarrayshavenotbeendescribedyetseeArraysofArrays.)
"@val_str_asc"
Orderbyelementvaluesinascendingorder(ratherthanbyindices).Scalarvaluesarecomparedasstrings.
Subarrays,ifpresent,comeoutlast.
"@val_num_asc"
152/479
29/7/2016
TheGNUAwkUsersGuide
Orderbyelementvaluesinascendingorder(ratherthanbyindices).Scalarvaluesarecomparedasnumbers.
Subarrays,ifpresent,comeoutlast.Whennumericvaluesareequal,thestringvaluesareusedtoprovidean
ordering:thisguaranteesconsistentresultsacrossdifferentversionsoftheCqsort()function,41whichgawkuses
internallytoperformthesorting.
"@ind_str_desc"
Like"@ind_str_asc",butthestringindicesareorderedfromhightolow.
"@ind_num_desc"
Like"@ind_num_asc",butthenumericindicesareorderedfromhightolow.
"@val_type_desc"
Like"@val_type_asc",buttheelementvalues,basedontype,areorderedfromhightolow.Subarrays,ifpresent,
comeoutfirst.
"@val_str_desc"
Like"@val_str_asc",buttheelementvalues,treatedasstrings,areorderedfromhightolow.Subarrays,if
present,comeoutfirst.
"@val_num_desc"
Like"@val_num_asc",buttheelementvalues,treatedasnumbers,areorderedfromhightolow.Subarrays,if
present,comeoutfirst.
Thearraytraversalorderisdeterminedbeforetheforloopstartstorun.ChangingPROCINFO["sorted_in"]intheloop
bodydoesnotaffecttheloop.Forexample:
$gawk'
>BEGIN{
>a[4]=4
>a[3]=3
>for(iina)
>printi,a[i]
>}'
|44
|33
$gawk'
>BEGIN{
>PROCINFO["sorted_in"]="@ind_str_asc"
>a[4]=4
>a[3]=3
>for(iina)
>printi,a[i]
>}'
|33
|44
Whensortinganarraybyelementvalues,ifavaluehappenstobeasubarraythenitisconsideredtobegreaterthanany
stringornumericvalue,regardlessofwhatthesubarrayitselfcontains,andallsubarraysaretreatedasbeingequalto
eachother.Theirorderrelativetoeachotherisdeterminedbytheirindexstrings.
Herearesomeadditionalthingstobearinmindaboutsortedarraytraversal:
ThevalueofPROCINFO["sorted_in"]isglobal.Thatis,itaffectsallarraytraversalforloops.Ifyouneedto
changeitwithinyourowncode,youshouldseeifitsdefinedandsaveandrestorethevalue:
if("sorted_in"inPROCINFO){
save_sorted=PROCINFO["sorted_in"]
PROCINFO["sorted_in"]="@val_str_desc"#orwhatever
153/479
29/7/2016
TheGNUAwkUsersGuide
if(save_sorted)
PROCINFO["sorted_in"]=save_sorted
Asalreadymentioned,thedefaultarraytraversalorderisrepresentedby"@unsorted".Youcanalsogetthedefault
behaviorbyassigningthenullstringtoPROCINFO["sorted_in"]orbyjustdeletingthe"sorted_in"elementfrom
thePROCINFOarraywiththedeletestatement.(ThedeletestatementhasntbeendescribedyetseeDelete.)
Inaddition,gawkprovidesbuiltinfunctionsforsortingarraysseeArraySortingFunctions.
Next:UninitializedSubscripts,Previous:ArrayBasics,Up:Arrays[Contents][Index]
8.2UsingNumberstoSubscriptArrays
Animportantaspecttorememberaboutarraysisthatarraysubscriptsarealwaysstrings.Whenanumericvalueisused
asasubscript,itisconvertedtoastringvaluebeforebeingusedforsubscripting(seeConversion).Thismeansthatthe
valueofthepredefinedvariableCONVFMTcanaffecthowyourprogramaccesseselementsofanarray.Forexample:
xyz=12.153
data[xyz]=1
CONVFMT="%2.2f"
if(xyzindata)
printf"%sisindata\n",xyz
else
printf"%sisnotindata\n",xyz
Thisprints12.15isnotindata.Thefirststatementgivesxyzanumericvalue.Assigningtodata[xyz]subscripts
datawiththestringvalue"12.153"(usingthedefaultconversionvalueofCONVFMT,"%.6g").Thus,thearrayelement
data["12.153"]isassignedthevalueone.TheprogramthenchangesthevalueofCONVFMT.Thetest(xyzindata)
generatesanewstringvaluefromxyzthistime"12.15"becausethevalueofCONVFMTonlyallowstwosignificant
digits.Thistestfails,because"12.15"isdifferentfrom"12.153".
Accordingtotherulesforconversions(seeConversion),integervaluesalwaysconverttostringsasintegers,nomatter
whatthevalueofCONVFMTmayhappentobe.Sotheusualcaseofthefollowingworks:
for(i=1;i<=maxsub;i++)
dosomethingwitharray[i]
Theintegervaluesalwaysconverttostringsasintegersrulehasanadditionalconsequenceforarrayindexing.Octal
andhexadecimalconstants(seeNondecimalnumbers)areconvertedinternallyintonumbers,andtheiroriginalformis
forgotten.Thismeans,forexample,thatarray[17],array[021],andarray[0x11]allrefertothesameelement!
Aswithmanythingsinawk,themajorityofthetimethingsworkasyouwouldexpectthemto.Butitisusefultohavea
preciseknowledgeoftheactualrules,astheycansometimeshaveasubtleeffectonyourprograms.
Next:Delete,Previous:NumericArraySubscripts,Up:Arrays[Contents][Index]
8.3UsingUninitializedVariablesasSubscripts
Supposeitsnecessarytowriteaprogramtoprinttheinputdatainreverseorder.Areasonableattempttodoso(with
sometestdata)mightlooklikethis:
$echo'line1
>line2
>line3'|awk'{l[lines]=$0;++lines}
>END{
>for(i=lines1;i>=0;i)
>printl[i]
>}'
154/479
29/7/2016
TheGNUAwkUsersGuide
|line3
|line2
Unfortunately,theveryfirstlineofinputdatadidnotappearintheoutput!
Uponfirstglance,wewouldthinkthatthisprogramshouldhaveworked.Thevariablelinesisuninitialized,and
uninitializedvariableshavethenumericvaluezero.So,awkshouldhaveprintedthevalueofl[0].
Theissuehereisthatsubscriptsforawkarraysarealwaysstrings.Uninitializedvariables,whenusedasstrings,havethe
value"",notzero.Thus,line1endsupstoredinl[""].Thefollowingversionoftheprogramworkscorrectly:
{l[lines++]=$0}
END{
for(i=lines1;i>=0;i)
printl[i]
}
Here,the++forceslinestobenumeric,thusmakingtheoldvaluenumericzero.Thisisthenconvertedto"0"asthe
arraysubscript.
Eventhoughitissomewhatunusual,thenullstring("")isavalidarraysubscript.(d.c.)gawkwarnsabouttheuseofthe
nullstringasasubscriptiflintisprovidedonthecommandline(seeOptions).
Next:Multidimensional,Previous:UninitializedSubscripts,Up:Arrays[Contents][Index]
8.4ThedeleteStatement
Toremoveanindividualelementofanarray,usethedeletestatement:
deletearray[indexexpression]
Onceanarrayelementhasbeendeleted,anyvaluetheelementoncehadisnolongeravailable.Itisasiftheelement
hadneverbeenreferredtoorbeengivenavalue.Thefollowingisanexampleofdeletingelementsinanarray:
for(iinfrequencies)
deletefrequencies[i]
Thisexampleremovesalltheelementsfromthearrayfrequencies.Onceanelementisdeleted,asubsequentfor
statementtoscanthearraydoesnotreportthatelementandusingtheinoperatortocheckforthepresenceofthat
elementreturnszero(i.e.,false):
deletefoo[4]
if(4infoo)
print"Thiswillneverbeprinted"
Itisimportanttonotethatdeletinganelementisnotthesameasassigningitanullvalue(theemptystring,"").For
example:
foo[4]=""
if(4infoo)
print"Thisisprinted,eventhoughfoo[4]isempty"
Itisnotanerrortodeleteanelementthatdoesnotexist.However,iflintisprovidedonthecommandline(see
Options),gawkissuesawarningmessagewhenanelementthatisnotinthearrayisdeleted.
Alltheelementsofanarraymaybedeletedwithasinglestatementbyleavingoffthesubscriptinthedeletestatement,
asfollows:
deletearray
Usingthisversionofthedeletestatementisaboutthreetimesmoreefficientthantheequivalentloopthatdeleteseach
155/479
29/7/2016
TheGNUAwkUsersGuide
elementoneatatime.
ThisformofthedeletestatementisalsosupportedbyBWKawkandmawk,aswellasbyanumberofother
implementations.
NOTE:Formanyyears,usingdeletewithoutasubscriptwasacommonextension.InSeptember2012,it
wasacceptedforinclusionintothePOSIXstandard.SeetheAustinGroupwebsite.
Thefollowingstatementprovidesaportablebutnonobviouswaytoclearoutanarray:42
split("",array)
Thesplit()function(seeStringFunctions)clearsoutthetargetarrayfirst.Thiscallasksittosplitapartthenullstring.
Becausethereisnodatatosplitout,thefunctionsimplyclearsthearrayandthenreturns.
CAUTION:Deletingalltheelementsfromanarraydoesnotchangeitstypeyoucannotclearanarrayand
thenusethearraysnameasascalar(i.e.,aregularvariable).Forexample,thefollowingdoesnotwork:
a[1]=3
deletea
a=3
Next:ArraysofArrays,Previous:Delete,Up:Arrays[Contents][Index]
8.5MultidimensionalArrays
Multiscanning: Scanningmultidimensionalarrays.
Amultidimensionalarrayisanarrayinwhichanelementisidentifiedbyasequenceofindicesinsteadofasingle
index.Forexample,atwodimensionalarrayrequirestwoindices.Theusualway(inmanylanguages,includingawk)to
refertoanelementofatwodimensionalarraynamedgridiswithgrid[x,y].
Multidimensionalarraysaresupportedinawkthroughconcatenationofindicesintoonestring.awkconvertstheindices
intostrings(seeConversion)andconcatenatesthemtogether,withaseparatorbetweenthem.Thiscreatesasinglestring
thatdescribesthevaluesoftheseparateindices.Thecombinedstringisusedasasingleindexintoanordinary,one
dimensionalarray.TheseparatorusedisthevalueofthebuiltinvariableSUBSEP.
Forexample,supposeweevaluatetheexpressionfoo[5,12]="value"whenthevalueof SUBSEPis"@".Thenumbers
5and12areconvertedtostringsandconcatenatedwithan@betweenthem,yielding"5@12"thus,thearrayelement
foo["5@12"]issetto"value".
Oncetheelementsvalueisstored,awkhasnorecordofwhetheritwasstoredwithasingleindexorasequenceof
indices.Thetwoexpressionsfoo[5,12]andfoo[5SUBSEP12]arealwaysequivalent.
ThedefaultvalueofSUBSEPisthestring"\034",whichcontainsanonprintingcharacterthatisunlikelytoappearinan
awkprogramorinmostinputdata.Theusefulnessofchoosinganunlikelycharactercomesfromthefactthatindex
valuesthatcontainastringmatchingSUBSEPcanleadtocombinedstringsthatareambiguous.SupposethatSUBSEPis
"@"thenfoo["a@b","c"]andfoo["a","b@c"]areindistinguishablebecausebothareactuallystoredas
foo["a@b@c"].
Totestwhetheraparticularindexsequenceexistsinamultidimensionalarray,usethesameoperator(in)thatisusedfor
singledimensionalarrays.Writethewholesequenceofindicesinparentheses,separatedbycommas,astheleft
operand:
if((subscript1,subscript2,)inarray)
Hereisanexamplethattreatsitsinputasatwodimensionalarrayoffieldsitrotatesthisarray90degreesclockwise
andprintstheresult.Itassumesthatalllineshavethesamenumberofelements:
156/479
29/7/2016
TheGNUAwkUsersGuide
{
if(max_nf<NF)
max_nf=NF
max_nr=NR
for(x=1;x<=NF;x++)
vector[x,NR]=$x
}
END{
for(x=1;x<=max_nf;x++){
for(y=max_nr;y>=1;y)
printf("%s",vector[x,y])
printf("\n")
}
}
Whengiventheinput:
123456
234561
345612
456123
theprogramproducesthefollowingoutput:
4321
5432
6543
1654
2165
3216
Up:Multidimensional[Contents][Index]
8.5.1ScanningMultidimensionalArrays
Thereisnospecialforstatementforscanningamultidimensionalarray.Therecannotbeone,because,intruth,awk
doesnothavemultidimensionalarraysorelementsthereisonlyamultidimensionalwayofaccessinganarray.
However,ifyourprogramhasanarraythatisalwaysaccessedasmultidimensional,youcangettheeffectofscanningit
bycombiningthescanningforstatement(seeScanninganArray)withthebuiltinsplit()function(seeString
Functions).Itworksinthefollowingmanner:
for(combinedinarray){
split(combined,separate,SUBSEP)
Thissetsthevariablecombinedtoeachconcatenatedcombinedindexinthearray,andsplitsitintotheindividualindices
bybreakingitapartwherethevalueofSUBSEPappears.Theindividualindicesthenbecometheelementsofthearray
separate.
Thus,ifavalueispreviouslystoredinarray[1,"foo"],thenanelementwithindex"1\034foo"existsinarray.(Recall
thatthedefaultvalueofSUBSEPisthecharacterwithcode034.)Soonerorlater,theforstatementfindsthatindexand
doesaniterationwiththevariablecombinedsetto"1\034foo".Thenthesplit()functioniscalledasfollows:
split("1\034foo",separate,"\034")
Theresultistosetseparate[1]to"1"andseparate[2]to"foo".Presto!Theoriginalsequenceofseparateindicesis
recovered.
Next:ArraysSummary,Previous:Multidimensional,Up:Arrays[Contents][Index]
157/479
29/7/2016
TheGNUAwkUsersGuide
8.6ArraysofArrays
gawkgoesbeyondstandardawksmultidimensionalarrayaccessandprovidestruearraysofarrays.Elementsofa
subarrayarereferredtobytheirownindicesenclosedinsquarebrackets,justliketheelementsofthemainarray.For
example,thefollowingcreatesatwoelementsubarrayatindex1ofthemainarraya:
a[1][1]=1
a[1][2]=2
Thissimulatesatruetwodimensionalarray.Eachsubarrayelementcancontainanothersubarrayasavalue,whichin
turncanholdotherarraysaswell.Inthisway,youcancreatearraysofthreeormoredimensions.Theindicescanbe
anyawkexpressions,includingscalarsseparatedbycommas(i.e.,aregularawksimulatedmultidimensionalsubscript).
Sothefollowingisvalidingawk:
a[1][3][1,"name"]="barney"
Eachsubarrayandthemainarraycanbeofdifferentlength.Infact,theelementsofanarrayoritssubarraydonotall
havetohavethesametype.Thismeansthatthemainarrayandanyofitssubarrayscanbenonrectangular,orjaggedin
structure.Youcanassignascalarvaluetotheindex4ofthemainarraya,eventhougha[1]isitselfanarrayandnota
scalar:
a[4]="Anelementinajaggedarray"
Thetermsdimension,row,andcolumnaremeaninglesswhenappliedtosuchanarray,butwewillusedimension
henceforthtoimplythemaximumnumberofindicesneededtorefertoanexistingelement.Thetypeofanyelement
thathasalreadybeenassignedcannotbechangedbyassigningavalueofadifferenttype.Youhavetofirstdeletethe
currentelement,whicheffectivelymakesgawkforgetabouttheelementatthatindex:
deletea[4]
a[4][5][6][7]="Anelementinafourdimensionalarray"
Thisremovesthescalarvaluefromindex4andtheninsertsathreelevelnestedsubarraycontainingascalar.Youcan
alsodeleteanentiresubarrayorsubarrayofsubarrays:
deletea[4][5]
a[4][5]="Anelementinsubarraya[4]"
Butrecallthatyoucannotdeletethemainarrayaandthenuseitasascalar.
Thebuiltinfunctionsthattakearrayargumentscanalsobeusedwithsubarrays.Forexample,thefollowingcode
fragmentuseslength()(seeStringFunctions)todeterminethenumberofelementsinthemainarrayaandits
subarrays:
printlength(a),length(a[1]),length(a[1][3])
Thisresultsinthefollowingoutputforourmainarraya:
2,3,1
Thesubscriptinarrayexpression(seeReferencetoElements)workssimilarlyforbothregularawkstylearraysand
arraysofarrays.Forexample,thetests1ina,3ina[1],and(1,"name")ina[1][3]allevaluatetoone(true)
forourarraya.
Thefor(iteminarray)statement(seeScanninganArray)canbenestedtoscanalltheelementsofanarrayof
arraysifitisrectangularinstructure.Inordertoprintthecontents(scalarvalues)ofatwodimensionalarrayofarrays
(i.e.,inwhicheachfirstlevelelementisitselfanarray,notnecessarilyofthesamelength),youcouldusethefollowing
code:
for(iinarray)
for(jinarray[i])
printarray[i][j]
158/479
29/7/2016
TheGNUAwkUsersGuide
Theisarray()function(seeTypeFunctions)letsyoutestifanarrayelementisitselfanarray:
for(iinarray){
if(isarray(array[i]){
for(jinarray[i]){
printarray[i][j]
}
}
else
printarray[i]
}
Ifthestructureofajaggedarrayofarraysisknowninadvance,youcanoftendeviseworkaroundsusingcontrol
statements.Forexample,thefollowingcodeprintstheelementsofourmainarraya:
for(iina){
for(jina[i]){
if(j==3){
for(kina[i][j])
printa[i][j][k]
}else
printa[i][j]
}
}
SeeWalkingArrays,forauserdefinedfunctionthatwalksanarbitrarilydimensionedarrayofarrays.
Recallthatareferencetoanuninitializedarrayelementyieldsavalueof"",thenullstring.Thishasoneimportant
implicationwhenyouintendtouseasubarrayasanargumenttoafunction,asillustratedbythefollowingexample:
$gawk'BEGIN{split("abcd",b[1]);printb[1][1]}'
errorgawk:cmd.line:1:fatal:split:secondargumentisnotanarray
Thewaytoworkaroundthisistofirstforceb[1]tobeanarraybycreatinganarbitraryindex:
$gawk'BEGIN{b[1][1]="";split("abcd",b[1]);printb[1][1]}'
|a
Previous:ArraysofArrays,Up:Arrays[Contents][Index]
8.7Summary
Standardawkprovidesonedimensionalassociativearrays(arraysindexedbystringvalues).Allarraysare
associativenumericindicesareconvertedautomaticallytostrings.
Arrayelementsarereferencedasarray[indx].Referencinganelementcreatesitifitdidnotexistpreviously.
Theproperwaytoseeifanarrayhasanelementwithagivenindexistousetheinoperator:indxinarray.
Usefor(indxinarray)toscanthroughalltheindividualelementsofanarray.Inthebodyoftheloop,indx
takesonthevalueofeachelementsindexinturn.
Theorderinwhichafor(indxinarray)looptraversesanarrayisundefinedinPOSIXawkandvariesamong
implementations.gawkletsyoucontroltheorderbyassigningspecialpredefinedvaluesto
PROCINFO["sorted_in"].
Usedeletearray[indx]todeleteanindividualelement.Todeletealloftheelementsinanarray,usedelete
array.Thislatterfeaturehasbeenacommonextensionformanyyearsandisnowstandard,butmaynotbe
supportedbyallcommercialversionsofawk.
Standardawksimulatesmultidimensionalarraysbyseparatingsubscriptvalueswithcommas.Thevaluesare
concatenatedintoasinglestring,separatedbythevalueofSUBSEP.Thefactthatsuchasubscriptwascreatedin
thiswayisnotretainedthus,changingSUBSEPmayhaveunexpectedconsequences.Youcanuse(sub1,sub2,
)inarraytoseeifsuchamultidimensionalsubscriptexistsinarray.
gawkprovidestruearraysofarrays.Youuseaseparatesetofsquarebracketsforeachdimensioninsuchanarray:
data[row][col],forexample.Arrayelementsmaythusbeeitherscalarvalues(numberorstring)orotherarrays.
Usetheisarray()builtinfunctiontodetermineifanarrayelementisitselfasubarray.
159/479
29/7/2016
TheGNUAwkUsersGuide
Next:LibraryFunctions,Previous:Arrays,Up:Top[Contents][Index]
9Functions
Thischapterdescribesawksbuiltinfunctions,whichfallintothreecategories:numeric,string,andI/O.gawkprovides
additionalgroupsoffunctionstoworkwithvaluesthatrepresenttime,dobitmanipulation,sortarrays,providetype
information,andinternationalizeandlocalizeprograms.
Besidesthebuiltinfunctions,awkhasprovisionsforwritingnewfunctionsthattherestofaprogramcanuse.The
secondhalfofthischapterdescribestheseuserdefinedfunctions.Finally,weexploreindirectfunctioncalls,agawk
specificextensionthatletsyoudetermineatruntimewhatfunctionistobecalled.
Builtin:
Summarizesthebuiltinfunctions.
Userdefined:
DescribesUserdefinedfunctionsindetail.
IndirectCalls:
Choosingthefunctiontocallatruntime.
FunctionsSummary: Summaryoffunctions.
Next:Userdefined,Up:Functions[Contents][Index]
9.1BuiltinFunctions
Builtinfunctionsarealwaysavailableforyourawkprogramtocall.Thissectiondefinesallthebuiltinfunctionsinawk
someofthesearementionedinothersectionsbutaresummarizedhereforyourconvenience.
CallingBuiltin:
Howtocallbuiltinfunctions.
NumericFunctions: Functionsthatworkwithnumbers,includingint(),sin()andrand().
StringFunctions: Functionsforstringmanipulation,suchassplit(),match()andsprintf().
I/OFunctions:
Functionsforfilesandshellcommands.
TimeFunctions:
Functionsfordealingwithtimestamps.
BitwiseFunctions: Functionsforbitwiseoperations.
TypeFunctions:
Functionsfortypeinformation.
I18NFunctions:
Functionsforstringtranslation.
Next:NumericFunctions,Up:Builtin[Contents][Index]
9.1.1CallingBuiltinFunctions
Tocalloneofawksbuiltinfunctions,writethenameofthefunctionfollowedbyargumentsinparentheses.For
example,atan2(y+z,1)isacalltothefunction atan2()andhastwoarguments.
Whitespaceisignoredbetweenthebuiltinfunctionnameandtheopeningparenthesis,butnonethelessitisgood
practicetoavoidusingwhitespacethere.Userdefinedfunctionsdonotpermitwhitespaceinthisway,anditiseasierto
avoidmistakesbyfollowingasimpleconventionthatalwaysworksnowhitespaceafterafunctionname.
Eachbuiltinfunctionacceptsacertainnumberofarguments.Insomecases,argumentscanbeomitted.Thedefaultsfor
omittedargumentsvaryfromfunctiontofunctionandaredescribedundertheindividualfunctions.Insomeawk
implementations,extraargumentsgiventobuiltinfunctionsareignored.However,ingawk,itisafatalerrortogive
extraargumentstoabuiltinfunction.
Whenafunctioniscalled,expressionsthatcreatethefunctionsactualparametersareevaluatedcompletelybeforethe
callisperformed.Forexample,inthefollowingcodefragment:
i=4
j=sqrt(i++)
160/479
29/7/2016
TheGNUAwkUsersGuide
thevariableiisincrementedtothevaluefivebeforesqrt()iscalledwithavalueoffourforitsactualparameter.The
orderofevaluationoftheexpressionsusedforthefunctionsparametersisundefined.Thus,avoidwritingprogramsthat
assumethatparametersareevaluatedfromlefttorightorfromrighttoleft.Forexample:
i=5
j=atan2(++i,i*=2)
Iftheorderofevaluationislefttoright,thenifirstbecomessix,andthen12,andatan2()iscalledwiththetwo
argumentssixand12.Butiftheorderofevaluationisrighttoleft,ifirstbecomes10,then11,andatan2()iscalled
withthetwoarguments11and10.
Next:StringFunctions,Previous:CallingBuiltin,Up:Builtin[Contents][Index]
9.1.2NumericFunctions
Thefollowinglistdescribesallofthebuiltinfunctionsthatworkwithnumbers.Optionalparametersareenclosedin
squarebrackets([]):
atan2(y,x)
Returnthearctangentofy/xinradians.Youcanusepi=atan2(0,1)toretrievethevalueofpi.
cos(x)
Returnthecosineofx,withxinradians.
exp(x)
Returntheexponentialofx(e^x)orreportanerrorifxisoutofrange.Therangeofvaluesxcanhavedepends
onyourmachinesfloatingpointrepresentation.
int(x)
Returnthenearestintegertox,locatedbetweenxandzeroandtruncatedtowardzero.Forexample,int(3)is3,
int(3.9)is3,int(3.9)is3,andint(3)is3aswell.
log(x)
Returnthenaturallogarithmofx,ifxispositiveotherwise,returnNaN(notanumber)onIEEE754systems.
Additionally,gawkprintsawarningmessagewhenxisnegative.
rand()
Returnarandomnumber.Thevaluesofrand()areuniformlydistributedbetweenzeroandone.Thevaluecould
bezerobutisneverone.43
Oftenrandomintegersareneededinstead.Followingisauserdefinedfunctionthatcanbeusedtoobtaina
randomnonnegativeintegerlessthann:
functionrandint(n)
{
returnint(n*rand())
}
Themultiplicationproducesarandomnumbergreaterthanorequaltozeroandlessthann.Usingint(),this
resultismadeintoanintegerbetweenzeroandn1,inclusive.
Thefollowingexampleusesasimilarfunctiontoproducerandomintegersbetweenoneandn.Thisprogram
printsanewrandomnumberforeachinputrecord:
161/479
29/7/2016
TheGNUAwkUsersGuide
#Functiontorollasimulateddie.
functionroll(n){return1+int(rand()*n)}
#Roll3sixsideddiceand
#printtotalnumberofpoints.
{
printf("%dpoints\n",roll(6)+roll(6)+roll(6))
}
CAUTION:Inmostawkimplementations,includinggawk,rand()startsgeneratingnumbersfromthe
samestartingnumber,orseed,eachtimeyourunawk.44Thus,aprogramgeneratesthesameresults
eachtimeyourunit.Thenumbersarerandomwithinoneawkrunbutpredictablefromruntorun.
Thisisconvenientfordebugging,butifyouwantaprogramtododifferentthingseachtimeitis
used,youmustchangetheseedtoavaluethatisdifferentineachrun.Todothis,usesrand().
sin(x)
Returnthesineofx,withxinradians.
sqrt(x)
Returnthepositivesquarerootofx.gawkprintsawarningmessageifxisnegative.Thus,sqrt(4)is2.
srand([x])
Setthestartingpoint,orseed,forgeneratingrandomnumberstothevaluex.
Eachseedvalueleadstoaparticularsequenceofrandomnumbers.45Thus,iftheseedissettothesamevaluea
secondtime,thesamesequenceofrandomnumbersisproducedagain.
CAUTION:Differentawkimplementationsusedifferentrandomnumbergeneratorsinternally.Dont
expectthesameawkprogramtoproducethesameseriesofrandomnumberswhenexecutedby
differentversionsofawk.
Iftheargumentxisomitted,asinsrand(),thenthecurrentdateandtimeofdayareusedforaseed.Thisisthe
waytogetrandomnumbersthataretrulyunpredictable.
Thereturnvalueofsrand()isthepreviousseed.Thismakesiteasytokeeptrackoftheseedsincaseyouneedto
consistentlyreproducesequencesofrandomnumbers.
POSIXdoesnotspecifytheinitialseeditdiffersamongawkimplementations.
Next:I/OFunctions,Previous:NumericFunctions,Up:Builtin[Contents][Index]
9.1.3StringManipulationFunctions
Thefunctionsinthissectionlookatorchangethetextofoneormorestrings.
gawkunderstandslocales(seeLocales)anddoesallstringprocessingintermsofcharacters,notbytes.Thisdistinction
isparticularlyimportanttounderstandforlocaleswhereonecharactermayberepresentedbymultiplebytes.Thus,for
example,length()returnsthenumberofcharactersinastring,andnotthenumberofbytesusedtorepresentthose
characters.Similarly,index()workswithcharacterindices,andnotbyteindices.
CAUTION:Anumberoffunctionsdealwithindicesintostrings.Forthesefunctions,thefirstcharacterof
astringisatposition(index)one.ThisisdifferentfromCandthelanguagesdescendedfromit,wherethe
firstcharacterisatpositionzero.Youneedtorememberthiswhendoingindexcalculations,particularlyif
youareusedtoC.
Inthefollowinglist,optionalparametersareenclosedinsquarebrackets([]).Severalfunctionsperformstring
162/479
29/7/2016
TheGNUAwkUsersGuide
substitutionthefulldiscussionisprovidedinthedescriptionofthesub()function,whichcomestowardtheend,
becausethelistispresentedalphabetically.
Thosefunctionsthatarespecifictogawkaremarkedwithapoundsign(#).Theyarenotavailableincompatibility
mode(seeOptions):
GoryDetails: Morethanyouwanttoknowabout\and&withsub(),gsub(),andgensub().
asort(source[,dest[,how]])#
asorti(source[,dest[,how]])#
Thesetwofunctionsaresimilarinbehavior,sotheyaredescribedtogether.
NOTE:Thefollowingdescriptionignoresthethirdargument,how,asitrequiresunderstanding
featuresthatwehavenotdiscussedyet.Thus,thediscussionhereisadeliberatesimplification.(We
doprovideallthedetailslateronseeArraySortingFunctions,forthefullstory.)
Bothfunctionsreturnthenumberofelementsinthearraysource.Forasort(),gawksortsthevaluesofsourceand
replacestheindicesofthesortedvaluesofsourcewithsequentialintegersstartingwithone.Iftheoptionalarray
destisspecified,thensourceisduplicatedintodest.destisthensorted,leavingtheindicesofsourceunchanged.
Whencomparingstrings,IGNORECASEaffectsthesorting(seeArraySortingFunctions).Ifthesourcearray
containssubarraysasvalues(seeArraysofArrays),theywillcomelast,afterallscalarvalues.Subarraysarenot
recursivelysorted.
Forexample,ifthecontentsofaareasfollows:
a["last"]="de"
a["first"]="sac"
a["middle"]="cul"
Acalltoasort():
asort(a)
resultsinthefollowingcontentsofa:
a[1]="cul"
a[2]="de"
a[3]="sac"
Theasorti()functionworkssimilarlytoasort()however,theindicesaresorted,insteadofthevalues.Thus,in
thepreviousexample,startingwiththesameinitialsetofindicesandvaluesina,callingasorti(a)wouldyield:
a[1]="first"
a[2]="last"
a[3]="middle"
gensub(regexp,replacement,how[,target])#
Searchthetargetstringtargetformatchesoftheregularexpressionregexp.Ifhowisastringbeginningwithg
orG(shortforglobal),thenreplaceallmatchesofregexpwithreplacement.Otherwise,howistreatedasa
numberindicatingwhichmatchofregexptoreplace.Ifnotargetissupplied,use$0.Itreturnsthemodifiedstring
astheresultofthefunctionandtheoriginaltargetstringisnotchanged.
gensub()isageneralsubstitutionfunction.Itspurposeistoprovidemorefeaturesthanthestandardsub()and
gsub()functions.
gensub()providesanadditionalfeaturethatisnotavailableinsub()orgsub():theabilitytospecifycomponents
ofaregexpinthereplacementtext.Thisisdonebyusingparenthesesintheregexptomarkthecomponentsand
thenspecifying\Ninthereplacementtext,whereNisadigitfrom1to9.Forexample:
163/479
29/7/2016
TheGNUAwkUsersGuide
$gawk'
>BEGIN{
>a="abcdef"
>b=gensub(/(.+)(.+)/,"\\2\\1","g",a)
>printb
>}'
|defabc
Aswithsub(),youmusttypetwobackslashesinordertogetoneintothestring.Inthereplacementtext,the
sequence\0representstheentirematchedtext,asdoesthecharacter&.
Thefollowingexampleshowshowyoucanusethethirdargumenttocontrolwhichmatchoftheregexpshould
bechanged:
$echoabcabc|
>gawk'{printgensub(/a/,"AA",2)}'
|abcAAbc
Inthiscase,$0isthedefaulttargetstring.gensub()returnsthenewstringasitsresult,whichispasseddirectlyto
printforprinting.
IfthehowargumentisastringthatdoesnotbeginwithgorG,orifitisanumberthatislessthanorequalto
zero,onlyonesubstitutionisperformed.Ifhowiszero,gawkissuesawarningmessage.
Ifregexpdoesnotmatchtarget,gensub()sreturnvalueistheoriginalunchangedvalueoftarget.
gsub(regexp,replacement[,target])
Searchtargetforallofthelongest,leftmost,nonoverlappingmatchingsubstringsitcanfindandreplacethem
withreplacement.Thegingsub()standsforglobal,whichmeansreplaceeverywhere.Forexample:
{gsub(/Britain/,"UnitedKingdom");print}
replacesalloccurrencesofthestringBritainwithUnitedKingdomforallinputrecords.
Thegsub()functionreturnsthenumberofsubstitutionsmade.Ifthevariabletosearchandalter(target)is
omitted,thentheentireinputrecord($0)isused.Asinsub(),thecharacters&and\arespecial,andthethird
argumentmustbeassignable.
index(in,find)
Searchthestringinforthefirstoccurrenceofthestringfind,andreturnthepositionincharacterswherethat
occurrencebeginsinthestringin.Considerthefollowingexample:
$awk'BEGIN{printindex("peanut","an")}'
|3
Iffindisnotfound,index()returnszero.
WithBWKawkandgawk,itisafatalerrortousearegexpconstantforfind.Otherimplementationsallowit,
simplytreatingtheregexpconstantasanexpressionmeaning$0~/regexp/.(d.c.)
length([string])
Returnthenumberofcharactersinstring.Ifstringisanumber,thelengthofthedigitstringrepresentingthat
numberisreturned.Forexample,length("abcde")isfive.Bycontrast,length(15*35)worksouttothree.In
thisexample,15*35=525,and525isthenconvertedtothestring"525",whichhasthreecharacters.
Ifnoargumentissupplied,length()returnsthelengthof$0.
NOTE:Inolderversionsofawk,thelength()functioncouldbecalledwithoutanyparentheses.
Doingsoisconsideredpoorpractice,althoughthe2008POSIXstandardexplicitlyallowsit,to
164/479
29/7/2016
TheGNUAwkUsersGuide
supporthistoricalpractice.Forprogramstobemaximallyportable,alwayssupplytheparentheses.
Iflength()iscalledwithavariablethathasnotbeenused,gawkforcesthevariabletobeascalar.Other
implementationsofawkleavethevariablewithoutatype.(d.c.)Consider:
$gawk'BEGIN{printlength(x);x[1]=1}'
|0
errorgawk:fatal:attempttousescalar`x'asarray
$nawk'BEGIN{printlength(x);x[1]=1}'
|0
Iflinthasbeenspecifiedonthecommandline,gawkissuesawarningaboutthis.
Withgawkandseveralotherawkimplementations,whengivenanarrayargument,thelength()functionreturns
thenumberofelementsinthearray.(c.e.)Thisislessusefulthanitmightseematfirst,asthearrayisnot
guaranteedtobeindexedfromonetothenumberofelementsinit.Iflintisprovidedonthecommandline
(seeOptions),gawkwarnsthatpassinganarrayargumentisnotportable.Ifposixissupplied,usinganarray
argumentisafatalerror(seeArrays).
match(string,regexp[,array])
Searchstringforthelongest,leftmostsubstringmatchedbytheregularexpressionregexpandreturnthecharacter
position(index)atwhichthatsubstringbegins(one,ifitstartsatthebeginningofstring).Ifnomatchisfound,
returnzero.
Theregexpargumentmaybeeitheraregexpconstant(//)orastringconstant("").Inthelattercase,the
stringistreatedasaregexptobematched.SeeComputedRegexps,foradiscussionofthedifferencebetweenthe
twoforms,andtheimplicationsforwritingyourprogramcorrectly.
Theorderofthefirsttwoargumentsistheoppositeofmostotherstringfunctionsthatworkwithregular
expressions,suchassub()andgsub().Itmighthelptorememberthatformatch(),theorderisthesameasforthe
~operator:string~regexp.
Thematch()functionsetsthepredefinedvariableRSTARTtotheindex.ItalsosetsthepredefinedvariableRLENGTH
tothelengthincharactersofthematchedsubstring.Ifnomatchisfound,RSTARTissettozero,andRLENGTHto1.
Forexample:
{
if($1=="FIND")
regex=$2
else{
where=match($0,regex)
if(where!=0)
print"Matchof",regex,"foundat",where,"in",$0
}
}
Thisprogramlooksforlinesthatmatchtheregularexpressionstoredinthevariableregex.Thisregular
expressioncanbechanged.IfthefirstwordonalineisFIND,regexischangedtobethesecondwordonthat
line.Therefore,ifgiven:
FINDru+n
Myprogramruns
butnotveryquickly
FINDMelvin
JF+KM
ThislineispropertyofRealityEngineeringCo.
Melvinwashere.
awkprints:
165/479
29/7/2016
TheGNUAwkUsersGuide
Matchofru+nfoundat12inMyprogramruns
MatchofMelvinfoundat1inMelvinwashere.
Ifarrayispresent,itiscleared,andthenthezerothelementofarrayissettotheentireportionofstringmatched
byregexp.Ifregexpcontainsparentheses,theintegerindexedelementsofarrayaresettocontaintheportionof
stringmatchingthecorrespondingparenthesizedsubexpression.Forexample:
$echofoooobazbarrrrr|
>gawk'{match($0,/(fo+).+(bar*)/,arr)
>printarr[1],arr[2]}'
|foooobarrrrr
Inaddition,multidimensionalsubscriptsareavailableprovidingthestartindexandlengthofeachmatched
subexpression:
$echofoooobazbarrrrr|
>gawk'{match($0,/(fo+).+(bar*)/,arr)
>printarr[1],arr[2]
>printarr[1,"start"],arr[1,"length"]
>printarr[2,"start"],arr[2,"length"]
>}'
|foooobarrrrr
|15
|97
Theremaynotbesubscriptsforthestartandindexforeveryparenthesizedsubexpression,becausetheymaynot
allhavematchedtextthus,theyshouldbetestedforwiththeinoperator(seeReferencetoElements).
Thearrayargumenttomatch()isagawkextension.Incompatibilitymode(seeOptions),usingathirdargumentis
afatalerror.
patsplit(string,array[,fieldpat[,seps]])#
Dividestringintopiecesdefinedbyfieldpatandstorethepiecesinarrayandtheseparatorstringsintheseps
array.Thefirstpieceisstoredinarray[1],thesecondpieceinarray[2],andsoforth.Thethirdargument,
fieldpat,isaregexpdescribingthefieldsinstring(justasFPATisaregexpdescribingthefieldsininputrecords).
Itmaybeeitheraregexpconstantorastring.Iffieldpatisomitted,thevalueofFPATisused.patsplit()returns
thenumberofelementscreated.seps[i]istheseparatorstringbetweenarray[i]andarray[i+1].Anyleading
separatorwillbeinseps[0].
Thepatsplit()functionsplitsstringsintopiecesinamannersimilartothewayinputlinesaresplitintofields
usingFPAT(seeSplittingByContent).
Beforesplittingthestring,patsplit()deletesanypreviouslyexistingelementsinthearraysarrayandseps.
split(string,array[,fieldsep[,seps]])
Dividestringintopiecesseparatedbyfieldsepandstorethepiecesinarrayandtheseparatorstringsintheseps
array.Thefirstpieceisstoredinarray[1],thesecondpieceinarray[2],andsoforth.Thestringvalueofthethird
argument,fieldsep,isaregexpdescribingwheretosplitstring(muchasFScanbearegexpdescribingwhereto
splitinputrecords).Iffieldsepisomitted,thevalueofFSisused.split()returnsthenumberofelementscreated.
sepsisagawkextension,withseps[i]beingtheseparatorstringbetweenarray[i]andarray[i+1].Iffieldsepisa
singlespace,thenanyleadingwhitespacegoesintoseps[0]andanytrailingwhitespacegoesintoseps[n],where
nisthereturnvalueofsplit()(i.e.,thenumberofelementsinarray).
Thesplit()functionsplitsstringsintopiecesinamannersimilartothewayinputlinesaresplitintofields.For
example:
split("culdesac",a,"",seps)
splitsthestring"culdesac"intothreefieldsusingastheseparator.Itsetsthecontentsofthearrayaas
follows:
166/479
29/7/2016
TheGNUAwkUsersGuide
a[1]="cul"
a[2]="de"
a[3]="sac"
andsetsthecontentsofthearraysepsasfollows:
seps[1]=""
seps[2]=""
Thevaluereturnedbythiscalltosplit()isthree.
Aswithinputfieldsplitting,whenthevalueoffieldsepis"",leadingandtrailingwhitespaceisignoredinvalues
assignedtotheelementsofarraybutnotinseps,andtheelementsareseparatedbyrunsofwhitespace.Also,as
withinputfieldsplitting,iffieldsepisthenullstring,eachindividualcharacterinthestringissplitintoitsown
arrayelement.(c.e.)
Note,however,thatRShasnoeffectonthewaysplit()works.EventhoughRS=""causesthenewline
charactertoalsobeaninputfieldseparator,thisdoesnotaffecthowsplit()splitsstrings.
Modernimplementationsofawk,includinggawk,allowthethirdargumenttobearegexpconstant(//)aswell
asastring.(d.c.)ThePOSIXstandardallowsthisaswell.SeeComputedRegexps,foradiscussionofthe
differencebetweenusingastringconstantoraregexpconstant,andtheimplicationsforwritingyourprogram
correctly.
Beforesplittingthestring,split()deletesanypreviouslyexistingelementsinthearraysarrayandseps.
Ifstringisnull,thearrayhasnoelements.(Sothisisaportablewaytodeleteanentirearraywithonestatement.
SeeDelete.)
Ifstringdoesnotmatchfieldsepatall(butisnotnull),arrayhasoneelementonly.Thevalueofthatelementis
theoriginalstring.
InPOSIXmode(seeOptions),thefourthargumentisnotallowed.
sprintf(format,expression1,)
Return(withoutprinting)thestringthatprintfwouldhaveprintedoutwiththesamearguments(seePrintf).For
example:
pival=sprintf("pi=%.2f(approx.)",22/7)
assignsthestringpi=3.14(approx.)tothevariablepival.
strtonum(str)#
Examinestrandreturnitsnumericvalue.Ifstrbeginswithaleading0,strtonum()assumesthatstrisanoctal
number.Ifstrbeginswithaleading0xor0X,strtonum()assumesthatstrisahexadecimalnumber.For
example:
$echo0x11|
>gawk'{printf"%d\n",strtonum($1)}'
|17
Usingthestrtonum()functionisnotthesameasaddingzerotoastringvaluetheautomaticcoercionofstringsto
numbersworksonlyfordecimaldata,notforoctalorhexadecimal.46
Notealsothatstrtonum()usesthecurrentlocalesdecimalpointforrecognizingnumbers(seeLocales).
sub(regexp,replacement[,target])
Searchtarget,whichistreatedasastring,fortheleftmost,longestsubstringmatchedbytheregularexpression
167/479
29/7/2016
TheGNUAwkUsersGuide
regexp.Modifytheentirestringbyreplacingthematchedtextwithreplacement.Themodifiedstringbecomesthe
newvalueoftarget.Returnthenumberofsubstitutionsmade(zeroorone).
Theregexpargumentmaybeeitheraregexpconstant(//)orastringconstant("").Inthelattercase,the
stringistreatedasaregexptobematched.SeeComputedRegexps,foradiscussionofthedifferencebetweenthe
twoforms,andtheimplicationsforwritingyourprogramcorrectly.
Thisfunctionispeculiarbecausetargetisnotsimplyusedtocomputeavalue,andnotjustanyexpressionwilldo
itmustbeavariable,field,orarrayelementsothatsub()canstoreamodifiedvaluethere.Ifthisargumentis
omitted,thenthedefaultistouseandalter$0.47Forexample:
str="water,water,everywhere"
sub(/at/,"ith",str)
setsstrtowither,water,everywhere,byreplacingtheleftmostlongestoccurrenceofatwithith.
Ifthespecialcharacter&appearsinreplacement,itstandsfortheprecisesubstringthatwasmatchedbyregexp.
(Iftheregexpcanmatchmorethanonestring,thenthisprecisesubstringmayvary.)Forexample:
{sub(/candidate/,"&andhiswife");print}
changesthefirstoccurrenceofcandidatetocandidateandhiswifeoneachinputline.Hereisanother
example:
$awk'BEGIN{
>str="daabaaa"
>sub(/a+/,"C&C",str)
>printstr
>}'
|dCaaCbaaa
Thisshowshow&canrepresentanonconstantstringandalsoillustratestheleftmost,longestruleinregexp
matching(seeLeftmostLongest).
Theeffectofthisspecialcharacter(&)canbeturnedoffbyputtingabackslashbeforeitinthestring.Asusual,
toinsertonebackslashinthestring,youmustwritetwobackslashes.Therefore,write\\&inastringconstantto
includealiteral&inthereplacement.Forexample,thefollowingshowshowtoreplacethefirst|oneachline
withan&:
{sub(/\|/,"\\&");print}
Asmentioned,thethirdargumenttosub()mustbeavariable,field,orarrayelement.Someversionsofawkallow
thethirdargumenttobeanexpressionthatisnotanlvalue.Insuchacase,sub()stillsearchesforthepatternand
returnszeroorone,buttheresultofthesubstitution(ifany)isthrownawaybecausethereisnoplacetoputit.
Suchversionsofawkacceptexpressionslikethefollowing:
sub(/USA/,"UnitedStates","theUSAandCanada")
Forhistoricalcompatibility,gawkacceptssucherroneouscode.However,usinganyothernonchangeableobjectas
thethirdparametercausesafatalerrorandyourprogramwillnotrun.
Finally,iftheregexpisnotaregexpconstant,itisconvertedintoastring,andthenthevalueofthatstringis
treatedastheregexptomatch.
substr(string,start[,length])
Returnalengthcharacterlongsubstringofstring,startingatcharacternumberstart.Thefirstcharacterofastring
ischaracternumberone.48Forexample,substr("washington",5,3)returns"ing".
Iflengthisnotpresent,substr()returnsthewholesuffixofstringthatbeginsatcharacternumberstart.For
example,substr("washington",5)returns"ington".Thewholesuffixisalsoreturnediflengthisgreaterthanthe
168/479
29/7/2016
TheGNUAwkUsersGuide
numberofcharactersremaininginthestring,countingfromcharacterstart.
Ifstartislessthanone,substr()treatsitasifitwasone.(POSIXdoesntspecifywhattodointhiscase:BWK
awkactsthisway,andthereforegawkdoestoo.)Ifstartisgreaterthanthenumberofcharactersinthestring,
substr()returnsthenullstring.Similarly,iflengthispresentbutlessthanorequaltozero,thenullstringis
returned.
Thestringreturnedbysubstr()cannotbeassigned.Thus,itisamistaketoattempttochangeaportionofa
string,asshowninthefollowingexample:
string="abcdef"
#trytoget"abCDEf",won'twork
substr(string,3,3)="CDE"
Itisalsoamistaketousesubstr()asthethirdargumentofsub()orgsub():
gsub(/xyz/,"pdq",substr($0,5,20))#WRONG
(Somecommercialversionsofawktreatsubstr()asassignable,butdoingsoisnotportable.)
Ifyouneedtoreplacebitsandpiecesofastring,combinesubstr()withstringconcatenation,inthefollowing
manner:
string="abcdef"
string=substr(string,1,2)"CDE"substr(string,6)
tolower(string)
Returnacopyofstring,witheachuppercasecharacterinthestringreplacedwithitscorrespondinglowercase
character.Nonalphabeticcharactersareleftunchanged.Forexample,tolower("MiXeDcAsE123")returns"mixed
case123".
toupper(string)
Returnacopyofstring,witheachlowercasecharacterinthestringreplacedwithitscorrespondinguppercase
character.Nonalphabeticcharactersareleftunchanged.Forexample,toupper("MiXeDcAsE123")returns"MIXED
CASE123".
MatchingtheNullString
Inawk,the*operatorcanmatchthenullstring.Thisisparticularlyimportantforthesub(),gsub(),andgensub()
functions.Forexample:
$echoabc|awk'{gsub(/m*/,"X");print}'
|XaXbXcX
Althoughthismakesacertainamountofsense,itcanbesurprising.
Up:StringFunctions[Contents][Index]
9.1.3.1Moreabout\and&withsub(),gsub(),andgensub()
CAUTION:Thissubsubsectionhasbeenreportedtocauseheadaches.Youmightwanttoskipituponfirst
reading.
Whenusingsub(),gsub(),orgensub(),andtryingtogetliteralbackslashesandampersandsintothereplacementtext,
youneedtorememberthatthereareseverallevelsofescapeprocessinggoingon.
169/479
29/7/2016
TheGNUAwkUsersGuide
First,thereisthelexicallevel,whichiswhenawkreadsyourprogramandbuildsaninternalcopyofittoexecute.Then
thereistheruntimelevel,whichiswhenawkactuallyscansthereplacementstringtodeterminewhattogenerate.
Atbothlevels,awklooksforadefinedsetofcharactersthatcancomeafterabackslash.Atthelexicallevel,itlooksfor
theescapesequenceslistedinEscapeSequences.Thus,forevery\thatawkprocessesattheruntimelevel,youmust
typetwobackslashesatthelexicallevel.Whenacharacterthatisnotvalidforanescapesequencefollowsthe\,
BWKawkandgawkbothsimplyremovetheinitial\andputthenextcharacterintothestring.Thus,forexample,
"a\qb"istreatedas"aqb".
Attheruntimelevel,thevariousfunctionshandlesequencesof\and&differently.Thesituationis(sadly)somewhat
complex.Historically,thesub()andgsub()functionstreatedthetwocharactersequence\&speciallythissequence
wasreplacedinthegeneratedtextwithasingle&.Anyother\withinthereplacementstringthatdidnotprecedean
&waspassedthroughunchanged.ThisisillustratedinTable9.1.
Youtypesub()seessub()generates
\&&Thematchedtext
\\&\&Aliteral&
\\\&\&Aliteral&
\\\\&\\&Aliteral\&
\\\\\&\\&Aliteral\&
\\\\\\&\\\&Aliteral\\&
\\q\qAliteral\q
Table9.1:Historicalescapesequenceprocessingforsub()andgsub()
Thistableshowsthelexicallevelprocessing,whereanoddnumberofbackslashesbecomesanevennumberatthe
runtimelevel,aswellastheruntimeprocessingdonebysub().(Forthesakeofsimplicity,therestofthefollowing
tablesonlyshowthecaseofevennumbersofbackslashesenteredatthelexicallevel.)
Theproblemwiththehistoricalapproachisthatthereisnowaytogetaliteral\followedbythematchedtext.
SeveraleditionsofthePOSIXstandardattemptedtofixthisproblembutwerentsuccessful.Thedetailsareirrelevantat
thispointintime.
Atonepoint,thegawkmaintainersubmittedproposedtextforarevisedstandardthatrevertstorulesthatcorrespond
morecloselytotheoriginalexistingpractice.Theproposedruleshavespecialcasesthatmakeitpossibletoproducea
\precedingthematchedtext.ThisisshowninTable9.2.
\\\\\\&\\\&Aliteral\&
\\\\&\\&Aliteral\,followedbythematchedtext
\\&\&Aliteral&
\\q\qAliteral\q
\\\\\\\\
Table9.2:gawkrulesforsub()andbackslash
Inanutshell,attheruntimelevel,therearenowthreespecialsequencesofcharacters(\\\&,\\&,and\&)whereas
historicallytherewasonlyone.However,asinthehistoricalcase,any\thatisnotpartofoneofthesethreesequences
isnotspecialandappearsintheoutputliterally.
gawk3.0and3.1followtheserulesforsub()andgsub().ThePOSIXstandardtookmuchlongertoberevisedthanwas
expected.Inaddition,thegawkmaintainersproposalwaslostduringthestandardizationprocess.Thefinalrulesare
somewhatsimpler.Theresultsaresimilarexceptforonecase.
ThePOSIXrulesstatethat\&inthereplacementstringproducesaliteral&,\\producesaliteral\,and\
followedbyanythingelseisnotspecialthe\isplacedstraightintotheoutput.TheserulesarepresentedinTable9.3.
170/479
29/7/2016
TheGNUAwkUsersGuide
\\\\\\&\\\&Aliteral\&
\\\\&\\&Aliteral\,followedbythematchedtext
\\&\&Aliteral&
\\q\qAliteral\q
\\\\\\\
Table9.3:POSIXrulesforsub()andgsub()
Theonlycasewherethedifferenceisnoticeableisthelastone:\\\\isseenas \\andproduces\insteadof\\.
Startingwithversion3.1.4,gawkfollowedthePOSIXruleswhenposixwasspecified(seeOptions).Otherwise,it
continuedtofollowtheproposedrules,asthathadbeenitsbehaviorformanyyears.
Whenversion4.0.0wasreleased,thegawkmaintainermadethePOSIXrulesthedefault,breakingwelloveradecades
worthofbackwardcompatibility.49Needlesstosay,thiswasabadidea,andasofversion4.0.1,gawkresumedits
historicalbehavior,andonlyfollowsthePOSIXruleswhenposixisgiven.
Therulesforgensub()areconsiderablysimpler.Attheruntimelevel,whenevergawkseesa\,ifthefollowing
characterisadigit,thenthetextthatmatchedthecorrespondingparenthesizedsubexpressionisplacedinthegenerated
output.Otherwise,nomatterwhatcharacterfollowsthe\,itappearsinthegeneratedtextandthe\doesnot,as
showninTable9.4.
Youtypegensub()seesgensub()generates
&&Thematchedtext
\\&\&Aliteral&
\\\\\\Aliteral\
\\\\&\\&Aliteral\,thenthematchedtext
\\\\\\&\\\&Aliteral\&
\\q\qAliteralq
Table9.4:Escapesequenceprocessingforgensub()
Becauseofthecomplexityofthelexicalandruntimelevelprocessingandthespecialcasesforsub()andgsub(),we
recommendtheuseofgawkandgensub()whenyouhavetodosubstitutions.
Next:TimeFunctions,Previous:StringFunctions,Up:Builtin[Contents][Index]
9.1.4Input/OutputFunctions
Thefollowingfunctionsrelatetoinput/output(I/O).Optionalparametersareenclosedinsquarebrackets([]):
close(filename[,how])
Closethefilefilenameforinputoroutput.Alternatively,theargumentmaybeashellcommandthatwasusedfor
creatingacoprocess,orforredirectingtoorfromapipethenthecoprocessorpipeisclosed.SeeCloseFilesAnd
Pipes,formoreinformation.
Whenclosingacoprocess,itisoccasionallyusefultofirstcloseoneendofthetwowaypipeandthentoclosethe
other.Thisisdonebyprovidingasecondargumenttoclose().Thissecondargument(how)shouldbeoneofthe
twostringvalues"to"or"from",indicatingwhichendofthepipetoclose.Caseinthestringdoesnotmatter.See
TwowayI/O,whichdiscussesthisfeatureinmoredetailandgivesanexample.
Notethatthesecondargumenttoclose()isagawkextensionitisnotavailableincompatibilitymode(see
Options).
171/479
29/7/2016
TheGNUAwkUsersGuide
fflush([filename])
Flushanybufferedoutputassociatedwithfilename,whichiseitherafileopenedforwritingorashellcommand
forredirectingoutputtoapipeorcoprocess.
Manyutilityprogramsbuffertheiroutput(i.e.,theysaveinformationtowritetoadiskfileorthescreenin
memoryuntilthereisenoughforittobeworthwhiletosendthedatatotheoutputdevice).Thisisoftenmore
efficientthanwritingeverylittlebitofinformationassoonasitisready.However,sometimesitisnecessaryto
forceaprogramtoflushitsbuffers(i.e.,writetheinformationtoitsdestination,evenifabufferisnotfull).Thisis
thepurposeofthefflush()functiongawkalsobuffersitsoutput,andthefflush()functionforcesgawktoflush
itsbuffers.
BrianKernighanaddedfflush()tohisawkinApril1992.Fortwodecades,itwasacommonextension.In
December2012,itwasacceptedforinclusionintothePOSIXstandard.SeetheAustinGroupwebsite.
POSIXstandardizesfflush()asfollows:ifthereisnoargument,oriftheargumentisthenullstring(""),then
awkflushesthebuffersforallopenoutputfilesandpipes.
NOTE:Priortoversion4.0.2,gawkwouldflushonlythestandardoutputiftherewasnoargument,
andflushalloutputfilesandpipesiftheargumentwasthenullstring.Thiswaschangedinorderto
becompatiblewithBrianKernighansawk,inthehopethatstandardizingthisfeatureinPOSIX
wouldthenbeeasier(whichindeedprovedtobethecase).
Withgawk,youcanusefflush("/dev/stdout")ifyouwishtoflushonlythestandardoutput.
fflush()returnszeroifthebufferissuccessfullyflushedotherwise,itreturnsanonzerovalue.(gawkreturns1.)
Inthecasewhereallbuffersareflushed,thereturnvalueiszeroonlyifallbufferswereflushedsuccessfully.
Otherwise,itis1,andgawkwarnsabouttheproblemfilename.
gawkalsoissuesawarningmessageifyouattempttoflushafileorpipethatwasopenedforreading(suchaswith
getline),oriffilenameisnotanopenfile,pipe,orcoprocess.Insuchacase,fflush()returns1,aswell.
InteractiveVersusNoninteractiveBuffering
Asasidepoint,bufferingissuescanbeevenmoreconfusingifyourprogramisinteractive(i.e.,communicating
withausersittingatakeyboard).50
Interactiveprogramsgenerallylinebuffertheiroutput(i.e.,theywriteouteveryline).Noninteractiveprograms
waituntiltheyhaveafullbuffer,whichmaybemanylinesofoutput.Hereisanexampleofthedifference:
$awk'{print$1+$2}'
11
|2
23
|5
Ctrld
Eachlineofoutputisprintedimmediately.Comparethatbehaviorwiththisexample:
$awk'{print$1+$2}'|cat
11
23
Ctrld
|2
|5
Here,nooutputisprinteduntilaftertheCtrldistyped,becauseitisallbufferedandsentdownthepipetocat
inoneshot.
system(command)
172/479
29/7/2016
TheGNUAwkUsersGuide
Executetheoperatingsystemcommandcommandandthenreturntotheawkprogram.Returncommandsexit
status.
Forexample,ifthefollowingfragmentofcodeisputinyourawkprogram:
END{
system("date|mails'awkrundone'root")
}
thesystemadministratorissentmailwhentheawkprogramfinishesprocessinginputandbeginsitsendofinput
processing.
Notethatredirectingprintorprintfintoapipeisoftenenoughtoaccomplishyourtask.Ifyouneedtorunmany
commands,itismoreefficienttosimplyprintthemdownapipelinetotheshell:
while(morestufftodo)
printcommand|"/bin/sh"
close("/bin/sh")
However,ifyourawkprogramisinteractive,system()isusefulforrunninglargeselfcontainedprograms,suchas
ashelloraneditor.Someoperatingsystemscannotimplementthesystem()function.system()causesafatal
errorifitisnotsupported.
NOTE:Whensandboxisspecified,thesystem()functionisdisabled(seeOptions).
ControllingOutputBufferingwithsystem()
Thefflush()functionprovidesexplicitcontroloveroutputbufferingforindividualfilesandpipes.However,itsuseis
notportabletomanyolderawkimplementations.Analternativemethodtoflushoutputbuffersistocallsystem()with
anullstringasitsargument:
system("")#flushoutput
gawktreatsthisuseofthesystem()functionasaspecialcaseandissmartenoughnottorunashell(orothercommand
interpreter)withtheemptycommand.Therefore,withgawk,thisidiomisnotonlyuseful,itisalsoefficient.Although
thismethodshouldworkwithotherawkimplementations,itdoesnotnecessarilyavoidstartinganunnecessaryshell.
(Otherimplementationsmayonlyflushthebufferassociatedwiththestandardoutputandnotnecessarilyallbuffered
output.)
Ifyouthinkaboutwhataprogrammerexpects,itmakessensethatsystem()shouldflushanypendingoutput.The
followingprogram:
BEGIN{
print"firstprint"
system("echosystemecho")
print"secondprint"
}
mustprint:
firstprint
systemecho
secondprint
andnot:
systemecho
firstprint
secondprint
Ifawkdidnotflushitsbuffersbeforecallingsystem(),youwouldseethelatter(undesirable)output.
173/479
29/7/2016
TheGNUAwkUsersGuide
Next:BitwiseFunctions,Previous:I/OFunctions,Up:Builtin[Contents][Index]
9.1.5TimeFunctions
awkprogramsarecommonlyusedtoprocesslogfilescontainingtimestampinformation,indicatingwhenaparticular
logrecordwaswritten.Manyprogramslogtheirtimestampsintheformreturnedbythetime()systemcall,whichisthe
numberofsecondssinceaparticularepoch.OnPOSIXcompliantsystems,itisthenumberofsecondssince197001
0100:00:00UTC,notcountingleapseconds.51AllknownPOSIXcompliantsystemssupporttimestampsfrom0
through2^311,whichissufficienttorepresenttimesthrough2038011903:14:07UTC.Manysystemssupporta
widerrangeoftimestamps,includingnegativetimestampsthatrepresenttimesbeforetheepoch.
Inordertomakeiteasiertoprocesssuchlogfilesandtoproduceusefulreports,gawkprovidesthefollowingfunctions
forworkingwithtimestamps.TheyaregawkextensionstheyarenotspecifiedinthePOSIXstandard.52However,
recentversionsofmawk(seeOtherVersions)alsosupportthesefunctions.Optionalparametersareenclosedinsquare
brackets([]):
mktime(datespec)
Turndatespecintoatimestampinthesameformasisreturnedbysystime().Itissimilartothefunctionofthe
samenameinISOC.Theargument,datespec,isastringoftheform"YYYYMMDDHHMMSS[DST]".Thestring
consistsofsixorsevennumbersrepresenting,respectively,thefullyearincludingcentury,themonthfrom1to
12,thedayofthemonthfrom1to31,thehourofthedayfrom0to23,theminutefrom0to59,thesecondfrom
0to60,53andanoptionaldaylightsavingsflag.
Thevaluesofthesenumbersneednotbewithintherangesspecifiedforexample,anhourof1means1hour
beforemidnight.TheoriginzeroGregoriancalendarisassumed,withyear0precedingyear1andyear1
precedingyear0.Thetimeisassumedtobeinthelocaltimezone.Ifthedaylightsavingsflagispositive,the
timeisassumedtobedaylightsavingstimeifzero,thetimeisassumedtobestandardtimeandifnegative(the
default),mktime()attemptstodeterminewhetherdaylightsavingstimeisineffectforthespecifiedtime.
Ifdatespecdoesnotcontainenoughelementsoriftheresultingtimeisoutofrange,mktime()returns1.
strftime([format[,timestamp[,utcflag]]])
Formatthetimespecifiedbytimestampbasedonthecontentsoftheformatstringandreturntheresult.Itis
similartothefunctionofthesamenameinISOC.Ifutcflagispresentandiseithernonzeroornonnull,the
valueisformattedasUTC(CoordinatedUniversalTime,formerlyGMTorGreenwichMeanTime).Otherwise,
thevalueisformattedforthelocaltimezone.Thetimestampisinthesameformatasthevaluereturnedbythe
systime()function.Ifnotimestampargumentissupplied,gawkusesthecurrenttimeofdayasthetimestamp.
Withoutaformatargument,strftime()usesthevalueofPROCINFO["strftime"]astheformatstring(seeBuiltin
Variables).Thedefaultstringvalueis"%a%b%e%H:%M:%S%Z%Y".Thisformatstringproducesoutputthatis
equivalenttothatofthedateutility.YoucanassignanewvaluetoPROCINFO["strftime"]tochangethedefault
formatseethefollowinglistforthevariousformatdirectives.
systime()
Returnthecurrenttimeasthenumberofsecondssincethesystemepoch.OnPOSIXsystems,thisisthenumber
ofsecondssince1970010100:00:00UTC,notcountingleapseconds.Itmaybeadifferentnumberonother
systems.
Thesystime()functionallowsyoutocompareatimestampfromalogfilewiththecurrenttimeofday.Inparticular,it
iseasytodeterminehowlongagoaparticularrecordwaslogged.Italsoallowsyoutoproducelogrecordsusingthe
secondssincetheepochformat.
Themktime()functionallowsyoutoconvertatextualrepresentationofadateandtimeintoatimestamp.Thismakesit
easytodobefore/aftercomparisonsofdatesandtimes,particularlywhendealingwithdateandtimedatacomingfrom
anexternalsource,suchasalogfile.
174/479
29/7/2016
TheGNUAwkUsersGuide
Thestrftime()functionallowsyoutoeasilyturnatimestampintohumanreadableinformation.Itissimilarinnature
tothesprintf()function(seeStringFunctions),inthatitcopiesnonformatspecificationcharactersverbatimtothe
returnedstring,whilesubstitutingdateandtimevaluesforformatspecificationsintheformatstring.
strftime()isguaranteedbythe1999ISOCstandard54tosupportthefollowingdateformatspecifications:
%a
Thelocalesabbreviatedweekdayname.
%A
Thelocalesfullweekdayname.
%b
Thelocalesabbreviatedmonthname.
%B
Thelocalesfullmonthname.
%c
Thelocalesappropriatedateandtimerepresentation.(Thisis%A%B%d%T%Yinthe"C"locale.)
%C
Thecenturypartofthecurrentyear.Thisistheyeardividedby100andtruncatedtothenextlowerinteger.
%d
Thedayofthemonthasadecimalnumber(0131).
%D
Equivalenttospecifying%m/%d/%y.
%e
Thedayofthemonth,paddedwithaspaceifitisonlyonedigit.
%F
Equivalenttospecifying%Y%m%d.ThisistheISO8601dateformat.
%g
Theyearmodulo100oftheISO8601weeknumber,asadecimalnumber(0099).Forexample,January1,2012,
isinweek53of2011.Thus,theyearofitsISO8601weeknumberis2011,eventhoughitsyearis2012.
Similarly,December31,2012,isinweek1of2013.Thus,theyearofitsISOweeknumberis2013,eventhough
itsyearis2012.
%G
ThefullyearoftheISOweeknumber,asadecimalnumber.
%h
Equivalentto%b.
%H
175/479
29/7/2016
TheGNUAwkUsersGuide
Thehour(24hourclock)asadecimalnumber(0023).
%I
Thehour(12hourclock)asadecimalnumber(0112).
%j
Thedayoftheyearasadecimalnumber(001366).
%m
Themonthasadecimalnumber(0112).
%M
Theminuteasadecimalnumber(0059).
%n
Anewlinecharacter(ASCIILF).
%p
ThelocalesequivalentoftheAM/PMdesignationsassociatedwitha12hourclock.
%r
Thelocales12hourclocktime.(Thisis%I:%M:%S%pinthe"C"locale.)
%R
Equivalenttospecifying%H:%M.
%S
Thesecondasadecimalnumber(0060).
%t
ATABcharacter.
%T
Equivalenttospecifying%H:%M:%S.
%u
Theweekdayasadecimalnumber(17).Mondayisdayone.
%U
Theweeknumberoftheyear(withthefirstSundayasthefirstdayofweekone)asadecimalnumber(0053).
%V
Theweeknumberoftheyear(withthefirstMondayasthefirstdayofweekone)asadecimalnumber(0153).
ThemethodfordeterminingtheweeknumberisasspecifiedbyISO8601.(Towit:iftheweekcontaining
January1hasfourormoredaysinthenewyear,thenitisweekoneotherwiseitisweek53ofthepreviousyear
andthenextweekisweekone.)
%w
176/479
29/7/2016
TheGNUAwkUsersGuide
Theweekdayasadecimalnumber(06).Sundayisdayzero.
%W
Theweeknumberoftheyear(withthefirstMondayasthefirstdayofweekone)asadecimalnumber(0053).
%x
Thelocalesappropriatedaterepresentation.(Thisis%A%B%d%Yinthe"C"locale.)
%X
Thelocalesappropriatetimerepresentation.(Thisis%Tinthe"C"locale.)
%y
Theyearmodulo100asadecimalnumber(0099).
%Y
Thefullyearasadecimalnumber(e.g.,2015).
%z
Thetimezoneoffsetina+HHMMformat(e.g.,theformatnecessarytoproduceRFC822/RFC1036dateheaders).
%Z
Thetimezonenameorabbreviationnocharactersifnotimezoneisdeterminable.
%Ec%EC%Ex%EX%Ey%EY%Od%Oe%OH
%OI%Om%OM%OS%Ou%OU%OV%Ow%OW%Oy
Alternativerepresentationsforthespecificationsthatuseonlythesecondletter(%c,%C,andsoon).55(These
facilitatecompliancewiththePOSIXdateutility.)
%%
Aliteral%.
Ifaconversionspecifierisnotoneofthosejustlisted,thebehaviorisundefined.56
Forsystemsthatarenotyetfullystandardscompliant,gawksuppliesacopyofstrftime()fromtheGNUCLibrary.It
supportsallofthejustlistedformatspecifications.Ifthatversionisusedtocompilegawk(seeInstallation),thenthe
followingadditionalformatspecificationsareavailable:
%k
Thehour(24hourclock)asadecimalnumber(023).Singledigitnumbersarepaddedwithaspace.
%l
Thehour(12hourclock)asadecimalnumber(112).Singledigitnumbersarepaddedwithaspace.
%s
Thetimeasadecimaltimestampinsecondssincetheepoch.
Additionally,thealternativerepresentationsarerecognizedbuttheirnormalrepresentationsareused.
ThefollowingexampleisanawkimplementationofthePOSIXdateutility.Normally,thedateutilityprintsthecurrent
dateandtimeofdayinawellknownformat.However,ifyouprovideanargumenttoitthatbeginswitha+,date
177/479
29/7/2016
TheGNUAwkUsersGuide
copiesnonformatspecifiercharacterstothestandardoutputandinterpretsthecurrenttimeaccordingtotheformat
specifiersinthestring.Forexample:
$date'+Todayis%A,%B%d,%Y.'
|TodayisMonday,September22,2014.
Hereisthegawkversionofthedateutility.Ithasashellwrappertohandletheuoption,whichrequiresthatdaterun
asifthetimezoneissettoUTC:
#!/bin/sh
#
#dateapproximatethePOSIX'date'command
case$1in
u)TZ=UTC0#useUTC
exportTZ
shift;;
esac
gawk'BEGIN{
format=PROCINFO["strftime"]
exitval=0
if(ARGC>2)
exitval=1
elseif(ARGC==2){
format=ARGV[1]
if(format~/^\+/)
format=substr(format,2)#removeleading+
}
printstrftime(format)
exitexitval
}'"$@"
Next:TypeFunctions,Previous:TimeFunctions,Up:Builtin[Contents][Index]
9.1.6BitManipulationFunctions
Icanexplainitforyou,butIcantunderstanditforyou.
Anonymous
Manylanguagesprovidetheabilitytoperformbitwiseoperationsontwointegernumbers.Inotherwords,theoperation
isperformedoneachsuccessivepairofbitsintheoperands.ThreecommonoperationsarebitwiseAND,OR,and
XOR.TheoperationsaredescribedinTable9.5.
Bitoperator
|AND|OR|XOR
|+++++
Operands|0|1|0|1|0|1
++++++
0|00|01|01
1|01|11|10
Table9.5:Bitwiseoperations
Asyoucansee,theresultofanANDoperationis1onlywhenbothbitsare1.TheresultofanORoperationis1if
eitherbitis1.TheresultofanXORoperationis1ifeitherbitis1,butnotboth.Thenextoperationisthecomplement
thecomplementof1is0andthecomplementof0is1.Thus,thisoperationflipsallthebitsofagivenvalue.
Finally,twoothercommonoperationsaretoshiftthebitsleftorright.Forexample,ifyouhaveabitstring10111001
178/479
29/7/2016
TheGNUAwkUsersGuide
andyoushiftitrightbythreebits,youendupwith00010111.57Ifyoustartoveragainwith10111001andshiftitleft
bythreebits,youendupwith11001000.Thefollowinglistdescribesgawksbuiltinfunctionsthatimplementthe
bitwiseoperations.Optionalparametersareenclosedinsquarebrackets([]):
and(v1,v2[,])
ReturnthebitwiseANDofthearguments.Theremustbeatleasttwo.
compl(val)
Returnthebitwisecomplementofval.
lshift(val,count)
Returnthevalueofval,shiftedleftbycountbits.
or(v1,v2[,])
ReturnthebitwiseORofthearguments.Theremustbeatleasttwo.
rshift(val,count)
Returnthevalueofval,shiftedrightbycountbits.
xor(v1,v2[,])
ReturnthebitwiseXORofthearguments.Theremustbeatleasttwo.
Forallofthesefunctions,firstthedoubleprecisionfloatingpointvalueisconvertedtothewidestCunsignedinteger
type,thenthebitwiseoperationisperformed.IftheresultcannotberepresentedexactlyasaCdouble,leadingnonzero
bitsareremovedonebyoneuntilitcanberepresentedexactly.TheresultisthenconvertedbackintoaCdouble.(If
youdontunderstandthisparagraph,dontworryaboutit.)
Hereisauserdefinedfunction(seeUserdefined)thatillustratestheuseofthesefunctions:
#bits2strturnabyteintoreadableonesandzeros
functionbits2str(bits,data,mask)
{
if(bits==0)
return"0"
mask=1
for(;bits!=0;bits=rshift(bits,1))
data=(and(bits,mask)?"1":"0")data
while((length(data)%8)!=0)
data="0"data
returndata
}
BEGIN{
printf"123=%s\n",bits2str(123)
printf"0123=%s\n",bits2str(0123)
printf"0x99=%s\n",bits2str(0x99)
comp=compl(0x99)
printf"compl(0x99)=%#x=%s\n",comp,bits2str(comp)
shift=lshift(0x99,2)
printf"lshift(0x99,2)=%#x=%s\n",shift,bits2str(shift)
shift=rshift(0x99,2)
printf"rshift(0x99,2)=%#x=%s\n",shift,bits2str(shift)
}
Thisprogramproducesthefollowingoutputwhenrun:
179/479
29/7/2016
TheGNUAwkUsersGuide
$gawkftestbits.awk
|123=01111011
|0123=01010011
|0x99=10011001
|compl(0x99)=0xffffff66=11111111111111111111111101100110
|lshift(0x99,2)=0x264=0000001001100100
|rshift(0x99,2)=0x26=00100110
Thebits2str()functionturnsabinarynumberintoastring.Initializingmasktoonecreatesabinaryvaluewherethe
rightmostbitissettoone.Usingthismask,thefunctionrepeatedlycheckstherightmostbit.ANDingthemaskwiththe
valueindicateswhethertherightmostbitisoneornot.Ifso,a"1"isconcatenatedontothefrontofthestring.
Otherwise,a"0"isadded.Thevalueisthenshiftedrightbyonebitandtheloopcontinuesuntiltherearenomoreone
bits.
Iftheinitialvalueiszero,itreturnsasimple"0".Otherwise,attheend,itpadsthevaluewithzerostorepresent
multiplesof8bitquantities.Thisistypicalinmoderncomputers.
ThemaincodeintheBEGINruleshowsthedifferencebetweenthedecimalandoctalvaluesforthesamenumbers(see
Nondecimalnumbers),andthendemonstratestheresultsofthecompl(),lshift(),andrshift()functions.
Next:I18NFunctions,Previous:BitwiseFunctions,Up:Builtin[Contents][Index]
9.1.7GettingTypeInformation
gawkprovidesasinglefunctionthatletsyoudistinguishanarrayfromascalarvariable.Thisisnecessaryforwriting
codethattraverseseveryelementofanarrayofarrays(seeArraysofArrays).
isarray(x)
Returnatruevalueifxisanarray.Otherwise,returnfalse.
isarray()ismeantforuseintwocircumstances.Thefirstiswhentraversingamultidimensionalarray:youcantestif
anelementisitselfanarrayornot.Thesecondisinsidethebodyofauserdefinedfunction(notdiscussedyetseeUser
defined),totestifaparameterisanarrayornot.
NOTE:Usingisarray()atthegloballeveltotestvariablesmakesnosense.Becauseyouaretheone
writingtheprogram,youaresupposedtoknowifyourvariablesarearraysornot.Andinfact,duetothe
waygawkworks,ifyoupassthenameofavariablethathasnotbeenpreviouslyusedtoisarray(),gawk
endsupturningitintoascalar.
Previous:TypeFunctions,Up:Builtin[Contents][Index]
9.1.8StringTranslationFunctions
gawkprovidesfacilitiesforinternationalizingawkprograms.Theseincludethefunctionsdescribedinthefollowinglist.
Thedescriptionsherearepurposelybrief.SeeInternationalization,forthefullstory.Optionalparametersareenclosed
insquarebrackets([]):
bindtextdomain(directory[,domain])
Setthedirectoryinwhichgawkwilllookformessagetranslationfiles,incasetheywillnotorcannotbeplacedin
thestandardlocations(e.g.,duringtesting).Itreturnsthedirectoryinwhichdomainisbound.
ThedefaultdomainisthevalueofTEXTDOMAIN.Ifdirectoryisthenullstring(""),thenbindtextdomain()returns
thecurrentbindingforthegivendomain.
dcgettext(string[,domain[,category]])
180/479
29/7/2016
TheGNUAwkUsersGuide
Returnthetranslationofstringintextdomaindomainforlocalecategorycategory.Thedefaultvaluefordomain
isthecurrentvalueofTEXTDOMAIN.Thedefaultvalueforcategoryis"LC_MESSAGES".
dcngettext(string1,string2,number[,domain[,category]])
Returnthepluralformusedfornumberofthetranslationofstring1andstring2intextdomaindomainforlocale
categorycategory.string1istheEnglishsingularvariantofamessage,andstring2istheEnglishpluralvariantof
thesamemessage.ThedefaultvaluefordomainisthecurrentvalueofTEXTDOMAIN.Thedefaultvalueforcategory
is"LC_MESSAGES".
Next:IndirectCalls,Previous:Builtin,Up:Functions[Contents][Index]
9.2UserDefinedFunctions
Complicatedawkprogramscanoftenbesimplifiedbydefiningyourownfunctions.Userdefinedfunctionscanbecalled
justlikebuiltinones(seeFunctionCalls),butitisuptoyoutodefinethem(i.e.,totellawkwhattheyshoulddo).
DefinitionSyntax: Howtowritedefinitionsandwhattheymean.
FunctionExample: Anexamplefunctiondefinitionandwhatitdoes.
FunctionCaveats: Thingstowatchoutfor.
ReturnStatement: Specifyingthevalueafunctionreturns.
DynamicTyping: Howvariabletypescanchangeatruntime.
Next:FunctionExample,Up:Userdefined[Contents][Index]
9.2.1FunctionDefinitionSyntax
Itsentirelyfairtosaythattheawksyntaxforlocalvariabledefinitionsisappallinglyawful.
BrianKernighan
Definitionsoffunctionscanappearanywherebetweentherulesofanawkprogram.Thus,thegeneralformofanawk
programisextendedtoincludesequencesofrulesanduserdefinedfunctiondefinitions.Thereisnoneedtoputthe
definitionofafunctionbeforeallusesofthefunction.Thisisbecauseawkreadstheentireprogrambeforestartingto
executeanyofit.
Thedefinitionofafunctionnamednamelookslikethis:
functionname([parameterlist])
{
bodyoffunction
}
Here,nameisthenameofthefunctiontodefine.Avalidfunctionnameislikeavalidvariablename:asequenceof
letters,digits,andunderscoresthatdoesntstartwithadigit.Heretoo,onlythe52upperandlowercaseEnglishletters
maybeusedinafunctionname.Withinasingleawkprogram,anyparticularnamecanonlybeusedasavariable,array,
orfunction.
parameterlistisanoptionallistofthefunctionsargumentsandlocalvariablenames,separatedbycommas.Whenthe
functioniscalled,theargumentnamesareusedtoholdtheargumentvaluesgiveninthecall.
Afunctioncannothavetwoparameterswiththesamename,normayithaveaparameterwiththesamenameasthe
functionitself.
CAUTION:AccordingtothePOSIXstandard,functionparameterscannothavethesamenameasoneof
thespecialpredefinedvariables(seeBuiltinVariables),normayafunctionparameterhavethesamename
181/479
29/7/2016
TheGNUAwkUsersGuide
asanotherfunction.
Notallversionsofawkenforcetheserestrictions.gawkalwaysenforcesthefirstrestriction.Withposix
(seeOptions),italsoenforcesthesecondrestriction.
Localvariablesactliketheemptystringifreferencedwhereastringvalueisrequired,andlikezeroifreferencedwhere
anumericvalueisrequired.Thisisthesameasthebehaviorofregularvariablesthathaveneverbeenassignedavalue.
(ThereismoretounderstandaboutlocalvariablesseeDynamicTyping.)
Thebodyoffunctionconsistsofawkstatements.Itisthemostimportantpartofthedefinition,becauseitsayswhatthe
functionshouldactuallydo.Theargumentnamesexisttogivethebodyawaytotalkabouttheargumentslocal
variablesexisttogivethebodyplacestokeeptemporaryvalues.
Argumentnamesarenotdistinguishedsyntacticallyfromlocalvariablenames.Instead,thenumberofarguments
suppliedwhenthefunctioniscalleddetermineshowmanyargumentvariablesthereare.Thus,ifthreeargumentvalues
aregiven,thefirstthreenamesinparameterlistareargumentsandtherestarelocalvariables.
Itfollowsthatifthenumberofargumentsisnotthesameinallcallstothefunction,someofthenamesinparameter
listmaybeargumentsonsomeoccasionsandlocalvariablesonothers.Anotherwaytothinkofthisisthatomitted
argumentsdefaulttothenullstring.
Usuallywhenyouwriteafunction,youknowhowmanynamesyouintendtouseforargumentsandhowmanyyou
intendtouseaslocalvariables.Itisconventionaltoplacesomeextraspacebetweentheargumentsandthelocal
variables,inordertodocumenthowyourfunctionissupposedtobeused.
Duringexecutionofthefunctionbody,theargumentsandlocalvariablevalueshide,orshadow,anyvariablesofthe
samenamesusedintherestoftheprogram.Theshadowedvariablesarenotaccessibleinthefunctiondefinition,
becausethereisnowaytonamethemwhiletheirnameshavebeentakenawayfortheargumentsandlocalvariables.
Allothervariablesusedintheawkprogramcanbereferencedorsetnormallyinthefunctionsbody.
Theargumentsandlocalvariableslastonlyaslongasthefunctionbodyisexecuting.Oncethebodyfinishes,youcan
onceagainaccessthevariablesthatwereshadowedwhilethefunctionwasrunning.
Thefunctionbodycancontainexpressionsthatcallfunctions.Theycanevencallthisfunction,eitherdirectlyorbyway
ofanotherfunction.Whenthishappens,wesaythefunctionisrecursive.Theactofafunctioncallingitselfiscalled
recursion.
Allthebuiltinfunctionsreturnavaluetotheircaller.Userdefinedfunctionscandosoalso,usingthereturnstatement,
whichisdescribedindetailinReturnStatement.Manyofthesubsequentexamplesinthissectionusethereturn
statement.
Inmanyawkimplementations,includinggawk,thekeywordfunctionmaybeabbreviatedfunc.(c.e.)However,POSIX
onlyspecifiestheuseofthekeywordfunction.Thisactuallyhassomepracticalimplications.IfgawkisinPOSIX
compatibilitymode(seeOptions),thenthefollowingstatementdoesnotdefineafunction:
funcfoo(){a=sqrt($1);printa}
Instead,itdefinesarulethat,foreachrecord,concatenatesthevalueofthevariablefuncwiththereturnvalueofthe
functionfoo.Iftheresultingstringisnonnull,theactionisexecuted.Thisisprobablynotwhatisdesired.(awk
acceptsthisinputassyntacticallyvalid,becausefunctionsmaybeusedbeforetheyaredefinedinawkprograms.58)
Toensurethatyourawkprogramsareportable,alwaysusethekeywordfunctionwhendefiningafunction.
Next:FunctionCaveats,Previous:DefinitionSyntax,Up:Userdefined[Contents][Index]
9.2.2FunctionDefinitionExamples
Hereisanexampleofauserdefinedfunction,calledmyprint(),thattakesanumberandprintsitinaspecificformat:
182/479
29/7/2016
TheGNUAwkUsersGuide
functionmyprint(num)
{
printf"%6.3g\n",num
}
Toillustrate,hereisanawkrulethatusesourmyprint()function:
$3>0{myprint($3)}
Thisprogramprints,inourspecialformat,allthethirdfieldsthatcontainapositivenumberinourinput.Therefore,
whengiventhefollowinginput:
1.23.45.67.8
9.1011.1213.1415.16
17.1819.2021.2223.24
thisprogram,usingourfunctiontoformattheresults,prints:
5.6
21.2
Thisfunctiondeletesalltheelementsinanarray(recallthattheextrawhitespacesignifiesthestartofthelocalvariable
list):
functiondelarray(a,i)
{
for(iina)
deletea[i]
}
Whenworkingwitharrays,itisoftennecessarytodeletealltheelementsinanarrayandstartoverwithanewlistof
elements(seeDelete).Insteadofhavingtorepeatthisloopeverywherethatyouneedtoclearoutanarray,yourprogram
canjustcalldelarray().(Thisguaranteesportability.Theuseofdeletearraytodeletethecontentsofanentirearray
isarelativelyrecent59additiontothePOSIXstandard.)
Thefollowingisanexampleofarecursivefunction.Ittakesastringasaninputparameterandreturnsthestringin
reverseorder.Recursivefunctionsmustalwayshaveatestthatstopstherecursion.Inthiscase,therecursionterminates
whentheinputstringisalreadyempty:
functionrev(str)
{
if(str=="")
return""
return(rev(substr(str,2))substr(str,1,1))
}
Ifthisfunctionisinafilenamedrev.awk,itcanbetestedthisway:
$echo"Don'tPanic!"|
>gawke'{printrev($0)}'frev.awk
|!cinaPt'noD
TheCctime()functiontakesatimestampandreturnsitasastring,formattedinawellknownfashion.Thefollowing
exampleusesthebuiltinstrftime()function(seeTimeFunctions)tocreateanawkversionofctime():
#ctime.awk
#
#awkversionofCctime(3)function
functionctime(ts,format)
{
format="%a%b%e%H:%M:%S%Z%Y"
if(ts==0)
183/479
29/7/2016
TheGNUAwkUsersGuide
ts=systime()#usecurrenttimeasdefault
returnstrftime(format,ts)
}
Youmightthinkthatctime()couldusePROCINFO["strftime"]foritsformatstring.Thatwouldbeamistake,because
ctime()issupposedtoreturnthetimeformattedinastandardfashion,anduserlevelcodecouldhavechanged
PROCINFO["strftime"].
Next:ReturnStatement,Previous:FunctionExample,Up:Userdefined[Contents][Index]
9.2.3CallingUserDefinedFunctions
Callingafunctionmeanscausingthefunctiontorunanddoitsjob.Afunctioncallisanexpressionanditsvalueisthe
valuereturnedbythefunction.
CallingAFunction:
Dontusespaces.
VariableScope:
Controllingvariablescope.
PassByValue/Reference: Passingparameters.
Next:VariableScope,Up:FunctionCaveats[Contents][Index]
9.2.3.1WritingaFunctionCall
Afunctioncallconsistsofthefunctionnamefollowedbytheargumentsinparentheses.awkexpressionsarewhatyou
writeinthecallforthearguments.Eachtimethecallisexecuted,theseexpressionsareevaluated,andthevalues
becometheactualarguments.Forexample,hereisacalltofoo()withthreearguments(thefirstbeingastring
concatenation):
foo(xy,"lose",4*z)
CAUTION:Whitespacecharacters(spacesandTABs)arenotallowedbetweenthefunctionnameandthe
openingparenthesisoftheargumentlist.Ifyouwritewhitespacebymistake,awkmightthinkthatyoumean
toconcatenateavariablewithanexpressioninparentheses.However,itnoticesthatyouusedafunction
nameandnotavariablename,andreportsanerror.
Next:PassByValue/Reference,Previous:CallingAFunction,Up:FunctionCaveats[Contents][Index]
9.2.3.2ControllingVariableScope
Unlikeinmanylanguages,thereisnowaytomakeavariablelocaltoa{}blockinawk,butyoucanmakeavariable
localtoafunction.Itisgoodpracticetodosowheneveravariableisneededonlyinthatfunction.
Tomakeavariablelocaltoafunction,simplydeclarethevariableasanargumentaftertheactualfunctionarguments
(seeDefinitionSyntax).Lookatthefollowingexample,wherevariableiisaglobalvariableusedbybothfunctions
foo()andbar():
functionbar()
{
for(i=0;i<3;i++)
print"bar'si="i
}
functionfoo(j)
{
i=j+1
print"foo'si="i
bar()
184/479
29/7/2016
TheGNUAwkUsersGuide
print"foo'si="i
}
BEGIN{
i=10
print"top'si="i
foo(0)
print"top'si="i
}
Runningthisscriptproducesthefollowing,becausetheiinfunctionsfoo()andbar()andatthetoplevelrefertothe
samevariableinstance:
top'si=10
foo'si=1
bar'si=0
bar'si=1
bar'si=2
foo'si=3
top'si=3
Ifyouwantitobelocaltobothfoo()andbar(),doasfollows(theextraspacebeforeiisacodingconventionto
indicatethatiisalocalvariable,notanargument):
functionbar(i)
{
for(i=0;i<3;i++)
print"bar'si="i
}
functionfoo(j,i)
{
i=j+1
print"foo'si="i
bar()
print"foo'si="i
}
BEGIN{
i=10
print"top'si="i
foo(0)
print"top'si="i
}
Runningthecorrectedscriptproducesthefollowing:
top'si=10
foo'si=1
bar'si=0
bar'si=1
bar'si=2
foo'si=1
top'si=10
Besidesscalarvalues(stringsandnumbers),youmayalsohavelocalarrays.Byusingaparameternameasanarray,awk
treatsitasanarray,anditislocaltothefunction.Inaddition,recursivecallscreatenewarrays.Considerthisexample:
functionsome_func(p1,a)
{
if(p1++>3)
return
a[p1]=p1
some_func(p1)
185/479
29/7/2016
TheGNUAwkUsersGuide
printf("Atlevel%d,index%d%sfoundina\n",
p1,(p11),(p11)ina?"is":"isnot")
printf("Atlevel%d,index%d%sfoundina\n",
p1,p1,p1ina?"is":"isnot")
print""
}
BEGIN{
some_func(1)
}
Whenrun,thisprogramproducesthefollowingoutput:
Atlevel4,index3isnotfoundina
Atlevel4,index4isfoundina
Previous:VariableScope,Up:FunctionCaveats[Contents][Index]
9.2.3.3PassingFunctionArgumentsbyValueOrbyReference
Inawk,whenyoudeclareafunction,thereisnowaytodeclareexplicitlywhethertheargumentsarepassedbyvalueor
byreference.
Instead,thepassingconventionisdeterminedatruntimewhenthefunctioniscalled,accordingtothefollowingrule:if
theargumentisanarrayvariable,thenitispassedbyreference.Otherwise,theargumentispassedbyvalue.
Passinganargumentbyvaluemeansthatwhenafunctioniscalled,itisgivenacopyofthevalueofthisargument.The
callermayuseavariableastheexpressionfortheargument,butthecalledfunctiondoesnotknowthisitonlyknows
whatvaluetheargumenthad.Forexample,ifyouwritethefollowingcode:
foo="bar"
z=myfunc(foo)
thenyoushouldnotthinkoftheargumenttomyfunc()asbeingthevariablefoo.Instead,thinkoftheargumentasthe
stringvalue"bar".Ifthefunctionmyfunc()altersthevaluesofitslocalvariables,thishasnoeffectonanyother
variables.Thus,ifmyfunc()doesthis:
functionmyfunc(str)
{
printstr
str="zzz"
printstr
}
tochangeitsfirstargumentvariablestr,itdoesnotchangethevalueoffoointhecaller.Theroleoffooincalling
myfunc()endedwhenitsvalue("bar")wascomputed.Ifstralsoexistsoutsideofmyfunc(),thefunctionbodycannot
alterthisoutervalue,becauseitisshadowedduringtheexecutionofmyfunc()andcannotbeseenorchangedfrom
there.
However,whenarraysaretheparameterstofunctions,theyarenotcopied.Instead,thearrayitselfismadeavailablefor
directmanipulationbythefunction.Thisisusuallytermedcallbyreference.Changesmadetoanarrayparameterinside
thebodyofafunctionarevisibleoutsidethatfunction.
NOTE:Changinganarrayparameterinsideafunctioncanbeverydangerousifyoudonotwatchwhatyou
aredoing.Forexample:
186/479
29/7/2016
TheGNUAwkUsersGuide
functionchangeit(array,ind,nvalue)
{
array[ind]=nvalue
}
BEGIN{
a[1]=1;a[2]=2;a[3]=3
changeit(a,2,"two")
printf"a[1]=%s,a[2]=%s,a[3]=%s\n",
a[1],a[2],a[3]
}
printsa[1]=1,a[2]=two,a[3]=3,becausechangeit()stores"two"inthesecondelementofa.
Someawkimplementationsallowyoutocallafunctionthathasnotbeendefined.Theyonlyreportaproblemat
runtime,whentheprogramactuallytriestocallthefunction.Forexample:
BEGIN{
if(0)
foo()
else
bar()
}
functionbar(){}
#notethat`foo'isnotdefined
Becausetheifstatementwillneverbetrue,itisnotreallyaproblemthatfoo()hasnotbeendefined.Usually,though,
itisaproblemifaprogramcallsanundefinedfunction.
Iflintisspecified(seeOptions),gawkreportscallstoundefinedfunctions.
Someawkimplementationsgeneratearuntimeerrorifyouuseeitherthenextstatementorthenextfilestatement(see
NextStatement,andseeNextfileStatement)insideauserdefinedfunction.gawkdoesnothavethislimitation.
Next:DynamicTyping,Previous:FunctionCaveats,Up:Userdefined[Contents][Index]
9.2.4ThereturnStatement
Asseeninseveralearlierexamples,thebodyofauserdefinedfunctioncancontainareturnstatement.Thisstatement
returnscontroltothecallingpartoftheawkprogram.Itcanalsobeusedtoreturnavalueforuseintherestoftheawk
program.Itlookslikethis:
return[expression]
Theexpressionpartisoptional.Duemostlikelytoanoversight,POSIXdoesnotdefinewhatthereturnvalueisifyou
omittheexpression.Technicallyspeaking,thismakesthereturnedvalueundefined,andtherefore,unpredictable.In
practice,though,allversionsofawksimplyreturnthenullstring,whichactslikezeroifusedinanumericcontext.
Areturnstatementwithoutanexpressionisassumedattheendofeveryfunctiondefinition.So,ifcontrolreachesthe
endofthefunctionbody,thentechnicallythefunctionreturnsanunpredictablevalue.Inpractice,itreturnstheempty
string.awkdoesnotwarnyouifyouusethereturnvalueofsuchafunction.
Sometimes,youwanttowriteafunctionforwhatitdoes,notforwhatitreturns.Suchafunctioncorrespondstoavoid
functioninC,C++,orJava,ortoaprocedureinAda.Thus,itmaybeappropriatetonotreturnanyvaluesimplybear
inmindthatyoushouldnotbeusingthereturnvalueofsuchafunction.
Thefollowingisanexampleofauserdefinedfunctionthatreturnsavalueforthelargestnumberamongtheelements
ofanarray:
functionmaxelt(vec,i,ret)
{
187/479
29/7/2016
TheGNUAwkUsersGuide
for(iinvec){
if(ret==""||vec[i]>ret)
ret=vec[i]
}
returnret
}
Youcallmaxelt()withoneargument,whichisanarrayname.Thelocalvariablesiandretarenotintendedtobe
argumentsthereisnothingtostopyoufrompassingmorethanoneargumenttomaxelt()buttheresultswouldbe
strange.Theextraspacebeforeiinthefunctionparameterlistindicatesthatiandretarelocalvariables.Youshould
followthisconventionwhendefiningfunctions.
Thefollowingprogramusesthemaxelt()function.Itloadsanarray,callsmaxelt(),andthenreportsthemaximum
numberinthatarray:
functionmaxelt(vec,i,ret)
{
for(iinvec){
if(ret==""||vec[i]>ret)
ret=vec[i]
}
returnret
}
#Loadallfieldsofeachrecordintonums.
{
for(i=1;i<=NF;i++)
nums[NR,i]=$i
}
END{
printmaxelt(nums)
}
Giventhefollowinginput:
1523816
44352826
25629113962962100
64679981101
99385110225
theprogramreports(predictably)that99,385isthelargestvalueinthearray.
Previous:ReturnStatement,Up:Userdefined[Contents][Index]
9.2.5FunctionsandTheirEffectsonVariableTyping
awkisaveryfluidlanguage.Itispossiblethatawkcanttellifanidentifierrepresentsascalarvariableoranarrayuntil
runtime.Hereisanannotatedsampleprogram:
functionfoo(a)
{
a[1]=1#parameterisanarray
}
BEGIN{
b=1
foo(b)#invalid:fataltypemismatch
foo(x)#xuninitialized,becomesanarraydynamically
x=1#nownotallowed,runtimeerror
}
188/479
29/7/2016
TheGNUAwkUsersGuide
Inthisexample,thefirstcalltofoo()generatesafatalerror,soawkwillnotreporttheseconderror.Ifyoucommentout
thatcall,though,thenawkdoesreporttheseconderror.
Usually,suchthingsarentabigissue,butitsworthbeingawareofthem.
Next:FunctionsSummary,Previous:Userdefined,Up:Functions[Contents][Index]
9.3IndirectFunctionCalls
Thissectiondescribesanadvanced,gawkspecificextension.
Often,youmaywishtodeferthechoiceoffunctiontocalluntilruntime.Forexample,youmayhavedifferentkindsof
records,eachofwhichshouldbeprocesseddifferently.
Normally,youwouldhavetouseaseriesofifelsestatementstodecidewhichfunctiontocall.Byusingindirect
functioncalls,youcanspecifythenameofthefunctiontocallasastringvariable,andthencallthefunction.Letslook
atanexample.
Supposeyouhaveafilewithyourtestscoresfortheclassesyouaretaking,andyouwishtogetthesumandtheaverage
ofyourtestscores.Thefirstfieldistheclassname.Thefollowingfieldsarethefunctionstocalltoprocessthedata,up
toamarkerfielddata:.Followingthemarker,totheendoftherecord,arethevariousnumerictestscores.
Hereistheinitialfile:
Biology_101sumaveragedata:87.092.478.594.9
Chemistry_305sumaveragedata:75.298.394.788.2
English_401sumaveragedata:100.095.687.193.4
Toprocessthedata,youmightwriteinitially:
{
class=$1
for(i=2;$i!="data:";i++){
if($i=="sum")
sum()#processesthewholerecord
elseif($i=="average")
average()
#andsoon
}
}
Thisstyleofprogrammingworks,butcanbeawkward.Withindirectfunctioncalls,youtellgawktousethevalueofa
variableasthenameofthefunctiontocall.
Thesyntaxissimilartothatofaregularfunctioncall:anidentifierimmediatelyfollowedbyanopeningparenthesis,
anyarguments,andthenaclosingparenthesis,withtheadditionofaleading@character:
the_func="sum"
result=@the_func()#callsthesum()function
Hereisafullprogramthatprocessesthepreviouslyshowndata,usingindirectfunctioncalls:
#indirectcall.awkDemonstrateindirectfunctioncalls
#averagereturntheaverageofthevaluesinfields$first$last
functionaverage(first,last,sum,i)
{
sum=0;
for(i=first;i<=last;i++)
sum+=$i
189/479
29/7/2016
TheGNUAwkUsersGuide
returnsum/(lastfirst+1)
}
#sumreturnthesumofthevaluesinfields$first$last
functionsum(first,last,ret,i)
{
ret=0;
for(i=first;i<=last;i++)
ret+=$i
returnret
}
Thesetwofunctionsexpecttoworkonfieldsthus,theparametersfirstandlastindicatewhereinthefieldstostart
andend.Otherwise,theyperformtheexpectedcomputationsandarenotunusual:
#Foreachrecord,printtheclassnameandtherequestedstatistics
{
class_name=$1
gsub(/_/,"",class_name)#Replace_withspaces
#findstart
for(i=1;i<=NF;i++){
if($i=="data:"){
start=i+1
break
}
}
printf("%s:\n",class_name)
for(i=2;$i!="data:";i++){
the_function=$i
printf("\t%s:<%s>\n",$i,@the_function(start,NF)"")
}
print""
}
Thisisthemainprocessingforeachrecord.Itprintstheclassname(withunderscoresreplacedwithspaces).Itthen
findsthestartoftheactualdata,savingitinstart.Thelastpartofthecodeloopsthrougheachfunctionname(from$2
uptothemarker,data:),callingthefunctionnamedbythefield.Theindirectfunctioncallitselfoccursasaparameter
inthecalltoprintf.(Theprintfformatstringuses%sastheformatspecifiersothatwecanusefunctionsthatreturn
strings,aswellasnumbers.Notethattheresultfromtheindirectcallisconcatenatedwiththeemptystring,inorderto
forceittobeastringvalue.)
Hereistheresultofrunningtheprogram:
$gawkfindirectcall.awkclass_data1
|Biology101:
|sum:<352.8>
|average:<88.2>
|
|Chemistry305:
|sum:<356.4>
|average:<89.1>
|
|English401:
|sum:<376.1>
|average:<94.025>
Theabilitytouseindirectfunctioncallsismorepowerfulthanyoumaythinkatfirst.TheCandC++languagesprovide
functionpointers,whichareamechanismforcallingafunctionchosenatruntime.Oneofthemostwellknownuses
ofthisabilityistheCqsort()function,whichsortsanarrayusingthefamousquicksortalgorithm(seetheWikipedia
articleformoreinformation).Tousethisfunction,yousupplyapointertoacomparisonfunction.Thismechanism
allowsyoutosortarbitrarydatainanarbitraryfashion.
190/479
29/7/2016
TheGNUAwkUsersGuide
Wecandosomethingsimilarusinggawk,likethis:
#quicksort.awkQuicksortalgorithm,withusersupplied
#comparisonfunction
#quicksortC.A.R.Hoare'squicksortalgorithm.SeeWikipedia
#oralmostanyalgorithmsorcomputersciencetext.
functionquicksort(data,left,right,less_than,i,last)
{
if(left>=right)#donothingifarraycontainsfewer
return#thantwoelements
quicksort_swap(data,left,int((left+right)/2))
last=left
for(i=left+1;i<=right;i++)
if(@less_than(data[i],data[left]))
quicksort_swap(data,++last,i)
quicksort_swap(data,left,last)
quicksort(data,left,last1,less_than)
quicksort(data,last+1,right,less_than)
}
#quicksort_swaphelperfunctionforquicksort,shouldreallybeinline
functionquicksort_swap(data,i,j,temp)
{
temp=data[i]
data[i]=data[j]
data[j]=temp
}
Thequicksort()functionreceivesthedataarray,thestartingandendingindicestosort(leftandright),andthename
ofafunctionthatperformsalessthancomparison.Itthenimplementsthequicksortalgorithm.
Tomakeuseofthesortingfunction,wereturntoourpreviousexample.Thefirstthingtodoiswritesomecomparison
functions:
#num_ltdoanumericlessthancomparison
functionnum_lt(left,right)
{
return((left+0)<(right+0))
}
#num_gedoanumericgreaterthanorequaltocomparison
functionnum_ge(left,right)
{
return((left+0)>=(right+0))
}
Thenum_ge()functionisneededtoperformadescendingsortwhenusedtoperformalessthantest,itactuallydoes
theopposite(greaterthanorequalto),whichyieldsdatasortedindescendingorder.
Nextcomesasortingfunction.Itisparameterizedwiththestartingandendingfieldnumbersandthecomparison
function.Itbuildsanarraywiththedataandcallsquicksort()appropriately,andthenformatstheresultsasasingle
string:
#do_sortsortthedataaccordingto`compare'
#andreturnitasastring
functiondo_sort(first,last,compare,data,i,retval)
{
deletedata
for(i=1;first<=last;first++){
data[i]=$first
191/479
29/7/2016
TheGNUAwkUsersGuide
i++
}
quicksort(data,1,i1,compare)
retval=data[1]
for(i=2;iindata;i++)
retval=retval""data[i]
returnretval
}
Finally,thetwosortingfunctionscalldo_sort(),passinginthenamesofthetwocomparisonfunctions:
#sortsortthedatainascendingorderandreturnitasastring
functionsort(first,last)
{
returndo_sort(first,last,"num_lt")
}
#rsortsortthedataindescendingorderandreturnitasastring
functionrsort(first,last)
{
returndo_sort(first,last,"num_ge")
}
Hereisanextendedversionofthedatafile:
Biology_101sumaveragesortrsortdata:87.092.478.594.9
Chemistry_305sumaveragesortrsortdata:75.298.394.788.2
English_401sumaveragesortrsortdata:100.095.687.193.4
Finally,herearetheresultswhentheenhancedprogramisrun:
$gawkfquicksort.awkfindirectcall.awkclass_data2
|Biology101:
|sum:<352.8>
|average:<88.2>
|sort:<78.587.092.494.9>
|rsort:<94.992.487.078.5>
|
|Chemistry305:
|sum:<356.4>
|average:<89.1>
|sort:<75.288.294.798.3>
|rsort:<98.394.788.275.2>
|
|English401:
|sum:<376.1>
|average:<94.025>
|sort:<87.193.495.6100.0>
|rsort:<100.095.693.487.1>
Anotherexamplewhereindirectfunctionscallsareusefulcanbefoundinprocessingarrays.Thisisdescribedin
WalkingArrays.
Rememberthatyoumustsupplyaleading@infrontofanindirectfunctioncall.
Startingwithversion4.1.2ofgawk,indirectfunctioncallsmayalsobeusedwithbuiltinfunctionsandwithextension
functions(seeDynamicExtensions).Therearesomelimitationswhencallingbuiltinfunctionsindirectly,asfollows.
Youcannotpassaregularexpressionconstanttoabuiltinfunctionthroughanindirectfunctioncall.60This
appliestothesub(),gsub(),gensub(),match(),split()andpatsplit()functions.
Ifcallingsub()orgsub(),youmayonlypasstwoarguments,sincethosefunctionsareunusualinthattheyupdate
192/479
29/7/2016
TheGNUAwkUsersGuide
theirthirdargument.Thismeansthat$0willbeupdated.
gawkdoesitsbesttomakeindirectfunctioncallsefficient.Forexample,inthefollowingcase:
for(i=1;i<=n;i++)
@the_func()
gawklooksuptheactualfunctiontocallonlyonce.
Previous:IndirectCalls,Up:Functions[Contents][Index]
9.4Summary
awkprovidesbuiltinfunctionsandletsyoudefineyourownfunctions.
POSIXawkprovidesthreekindsofbuiltinfunctions:numeric,string,andI/O.gawkprovidesfunctionsthatsort
arrays,workwithvaluesrepresentingtime,dobitmanipulation,determinevariabletype(arrayversusscalar),and
internationalizeandlocalizeprograms.gawkalsoprovidesseveralextensionstosomeofstandardfunctions,
typicallyintheformofadditionalarguments.
Functionsacceptzeroormoreargumentsandreturnavalue.Theexpressionsthatprovidetheargumentvaluesare
completelyevaluatedbeforethefunctioniscalled.Orderofevaluationisnotdefined.Thereturnvaluecanbe
ignored.
Thehandlingofbackslashinsub()andgsub()isnotsimple.Itismorestraightforwardingawksgensub()
function,butthatfunctionstillrequirescareinitsuse.
Userdefinedfunctionsprovideimportantcapabilitiesbutcomewithsomesyntacticinelegancies.Inafunction
call,therecannotbeanyspacebetweenthefunctionnameandtheopeningleftparenthesisoftheargumentlist.
Also,thereisnoprovisionforlocalvariables,sotheconventionistoaddextraparameters,andtoseparatethem
visuallyfromtherealparametersbyextrawhitespace.
Userdefinedfunctionsmaycallotheruserdefined(andbuiltin)functionsandmaycallthemselvesrecursively.
Functionparametershideanyglobalvariablesofthesamenames.Youcannotusethenameofareserved
variable(suchasARGC)asthenameofaparameterinuserdefinedfunctions.
Scalarvaluesarepassedtouserdefinedfunctionsbyvalue.Arrayparametersarepassedbyreferenceany
changesmadebythefunctiontoarrayparametersarethusvisibleafterthefunctionhasreturned.
Usethereturnstatementtoreturnfromauserdefinedfunction.Anoptionalexpressionbecomesthefunctions
returnvalue.Onlyscalarvaluesmaybereturnedbyafunction.
Ifavariablethathasneverbeenusedispassedtoauserdefinedfunction,howthatfunctiontreatsthevariable
cansetitsnature:eitherscalarorarray.
gawkprovidesindirectfunctioncallsusingaspecialsyntax.Bysettingavariabletothenameofafunction,you
candetermineatruntimewhatfunctionwillbecalledatthatpointintheprogram.Thisisequivalenttofunction
pointersinCandC++.
Next:SamplePrograms,Previous:Functions,Up:Top[Contents][Index]
PartII:
Userdefined,describeshowtowriteyourownawkfunctions.Writingfunctionsisimportant,becauseitallowsyouto
encapsulatealgorithmsandprogramtasksinasingleplace.Itsimplifiesprogramming,makingprogramdevelopment
moremanageableandmakingprogramsmorereadable.
Intheirseminal1976book,SoftwareTools,61BrianKernighanandP.J.Plaugerwrote:
193/479
29/7/2016
TheGNUAwkUsersGuide
GoodProgrammingisnotlearnedfromgeneralities,butbyseeinghowsignificantprogramscanbemade
clean,easytoread,easytomaintainandmodify,humanengineered,efficientandreliable,bythe
applicationofcommonsenseandgoodprogrammingpractices.Carefulstudyandimitationofgood
programsleadstobetterwriting.
Infact,theyfeltthisideawassoimportantthattheyplacedthisstatementonthecoveroftheirbook.Becausewe
believestronglythattheirstatementiscorrect,thischapterandSamplePrograms,provideagoodsizedbodyofcodefor
youtoreadand,wehope,tolearnfrom.
Thischapterpresentsalibraryofusefulawkfunctions.ManyofthesampleprogramspresentedlaterinthisWebpage
usethesefunctions.Thefunctionsarepresentedhereinaprogressionfromsimpletocomplex.
ExtractProgram,presentsaprogramthatyoucanusetoextractthesourcecodefortheseexamplelibraryfunctionsand
programsfromtheTexinfosourceforthisWebpage.(Thishasalreadybeendoneaspartofthegawkdistribution.)
Ifyouhavewrittenoneormoreuseful,generalpurposeawkfunctionsandwouldliketocontributethemtotheawkuser
community,seeHowToContribute,formoreinformation.
TheprogramsinthischapterandinSamplePrograms,freelyusegawkspecificfeatures.Rewritingtheseprogramsfor
differentimplementationsofawkisprettystraightforward:
Diagnosticerrormessagesaresentto/dev/stderr.Use|"cat1>&2"insteadof>"/dev/stderr"ifyour
systemdoesnothavea/dev/stderr,orifyoucannotusegawk.
Anumberofprogramsusenextfile(seeNextfileStatement)toskipanyremaininginputintheinputfile.
Finally,someoftheprogramschoosetoignoreupperandlowercasedistinctionsintheirinput.Theydosoby
assigningonetoIGNORECASE.Youcanachievealmostthesameeffect62byaddingthefollowingruletothe
beginningoftheprogram:
#ignorecase
{$0=tolower($0)}
Also,verifythatallregexpandstringconstantsusedincomparisonsuseonlylowercaseletters.
LibraryNames:
Howtobestnameprivateglobalvariablesinlibraryfunctions.
GeneralFunctions:
Functionsthatareofgeneraluse.
DataFileManagement:
Functionsformanagingcommandlinedatafiles.
GetoptFunction:
Afunctionforprocessingcommandlinearguments.
PasswdFunctions:
Functionsforgettinguserinformation.
GroupFunctions:
Functionsforgettinggroupinformation.
WalkingArrays:
Afunctiontowalkarraysofarrays.
LibraryFunctionsSummary: Summaryoflibraryfunctions.
LibraryExercises:
Exercises.
Next:GeneralFunctions,Up:LibraryFunctions[Contents][Index]
10.1NamingLibraryFunctionGlobalVariables
Duetothewaytheawklanguageevolved,variablesareeitherglobal(usablebytheentireprogram)orlocal(usablejust
byaspecificfunction).ThereisnointermediatestateanalogoustostaticvariablesinC.
Libraryfunctionsoftenneedtohaveglobalvariablesthattheycanusetopreservestateinformationbetweencallstothe
functionforexample,getopt()svariable_opti(seeGetoptFunction).Suchvariablesarecalledprivate,astheonly
functionsthatneedtousethemaretheonesinthelibrary.
Whenwritingalibraryfunction,youshouldtrytochoosenamesforyourprivatevariablesthatwillnotconflictwith
anyvariablesusedbyeitheranotherlibraryfunctionorausersmainprogram.Forexample,anamelikeiorjisnota
goodchoice,becauseuserprogramsoftenusevariablenameslikethesefortheirownpurposes.
194/479
29/7/2016
TheGNUAwkUsersGuide
Theexampleprogramsshowninthischapterallstartthenamesoftheirprivatevariableswithanunderscore(_).Users
generallydontuseleadingunderscoresintheirvariablenames,sothisconventionimmediatelydecreasesthechances
thatthevariablenameswillbeaccidentallysharedwiththeusersprogram.
Inaddition,severalofthelibraryfunctionsuseaprefixthathelpsindicatewhatfunctionorsetoffunctionsusethe
variablesforexample,_pw_byname()intheuserdatabaseroutines(seePasswdFunctions).Thisconventionis
recommended,asitevenfurtherdecreasesthechanceofinadvertentconflictamongvariablenames.Notethatthis
conventionisusedequallywellforvariablenamesandforprivatefunctionnames.63
Asafinalnoteonvariablenaming,ifafunctionmakesglobalvariablesavailableforusebyamainprogram,itisagood
conventiontostartthosevariablesnameswithacapitalletterforexample,getopt()sOpterrandOptindvariables
(seeGetoptFunction).Theleadingcapitalletterindicatesthatitisglobal,whilethefactthatthevariablenameisnotall
capitallettersindicatesthatthevariableisnotoneofawkspredefinedvariables,suchasFS.
Itisalsoimportantthatallvariablesinlibraryfunctionsthatdonotneedtosavestateare,infact,declaredlocal.64If
thisisnotdone,thevariablescouldaccidentallybeusedintheusersprogram,leadingtobugsthatareverydifficultto
trackdown:
functionlib_func(x,y,l1,l2)
{
#some_varshouldbelocalbutbyoversightisnot
usevariablesome_var
Adifferentconvention,commonintheTclcommunity,istouseasingleassociativearraytoholdthevaluesneededby
thelibraryfunction(s),orpackage.Thissignificantlydecreasesthenumberofactualglobalnamesinuse.For
example,thefunctionsdescribedinPasswdFunctions,mighthaveusedarrayelementsPW_data["inited"],
PW_data["total"],PW_data["count"],andPW_data["awklib"],insteadof_pw_inited,_pw_awklib,_pw_total,and
_pw_count.
Theconventionspresentedinthissectionareexactlythat:conventions.Youarenotrequiredtowriteyourprogramsthis
waywemerelyrecommendthatyoudoso.
Next:DataFileManagement,Previous:LibraryNames,Up:LibraryFunctions[Contents][Index]
10.2GeneralProgramming
Thissectionpresentsanumberoffunctionsthatareofgeneralprogramminguse.
StrtonumFunction:
Areplacementforthebuiltinstrtonum()function.
AssertFunction:
Afunctionforassertionsinawkprograms.
RoundFunction:
Afunctionforroundingifsprintf()doesnotdoitcorrectly.
CliffRandomFunction: TheCliffRandomNumberGenerator.
OrdinalFunctions:
Functionsforusingcharactersasnumbersandviceversa.
JoinFunction:
Afunctiontojoinanarrayintoastring.
GetlocaltimeFunction: Afunctiontogetformattedtimes.
ReadfileFunction:
Afunctiontoreadanentirefileatonce.
ShellQuoting:
Afunctiontoquotestringsfortheshell.
Next:AssertFunction,Up:GeneralFunctions[Contents][Index]
10.2.1ConvertingStringstoNumbers
Thestrtonum()function(seeStringFunctions)isagawkextension.Thefollowingfunctionprovidesanimplementation
195/479
29/7/2016
TheGNUAwkUsersGuide
forotherversionsofawk:
#mystrtonumconvertstringtonumber
functionmystrtonum(str,ret,n,i,k,c)
{
if(str~/^0[07]*$/){
#octal
n=length(str)
ret=0
for(i=1;i<=n;i++){
c=substr(str,i,1)
#index()returns0ifcnotinstring,
#includesc=="0"
k=index("1234567",c)
ret=ret*8+k
}
}elseif(str~/^0[xX][[:xdigit:]]+$/){
#hexadecimal
str=substr(str,3)#lopoffleading0x
n=length(str)
ret=0
for(i=1;i<=n;i++){
c=substr(str,i,1)
c=tolower(c)
#index()returns0ifcnotinstring,
#includesc=="0"
k=index("123456789abcdef",c)
ret=ret*16+k
}
}elseif(str~\
/^[+]?([09]+([.][09]*([Ee][09]+)?)?|([.][09]+([Ee][+]?[09]+)?))$/){
#decimalnumber,possiblyfloatingpoint
ret=str+0
}else
ret="NOTANUMBER"
returnret
}
#BEGIN{#gawktestharness
#a[1]="25"
#a[2]=".31"
#a[3]="0123"
#a[4]="0xdeadBEEF"
#a[5]="123.45"
#a[6]="1.e3"
#a[7]="1.32"
#a[8]="1.32E2"
#
#for(i=1;iina;i++)
#printa[i],strtonum(a[i]),mystrtonum(a[i])
#}
ThefunctionfirstlooksforCstyleoctalnumbers(base8).Iftheinputstringmatchesaregularexpressiondescribing
octalnumbers,thenmystrtonum()loopsthrougheachcharacterinthestring.Itsetsktotheindexin"1234567"ofthe
currentoctaldigit.Thereturnvaluewilleitherbethesamenumberasthedigit,orzeroifthecharacterisnotthere,
whichwillbetruefora0.Thisissafe,becausetheregexptestintheifensuresthatonlyoctalvaluesareconverted.
Similarlogicappliestothecodethatchecksforandconvertsahexadecimalvalue,whichstartswith0xor0X.The
useoftolower()simplifiesthecomputationforfindingthecorrectnumericvalueforeachhexadecimaldigit.
Finally,ifthestringmatchesthe(rathercomplicated)regexpforaregulardecimalintegerorfloatingpointnumber,the
computationret=str+0letsawkconvertthevaluetoanumber.
196/479
29/7/2016
TheGNUAwkUsersGuide
Acommentedouttestprogramisincluded,sothatthefunctioncanbetestedwithgawkandtheresultscomparedtothe
builtinstrtonum()function.
Next:RoundFunction,Previous:StrtonumFunction,Up:GeneralFunctions[Contents][Index]
10.2.2Assertions
Whenwritinglargeprograms,itisoftenusefultoknowthataconditionorsetofconditionsistrue.Beforeproceeding
withaparticularcomputation,youmakeastatementaboutwhatyoubelievetobethecase.Suchastatementisknown
asanassertion.TheClanguageprovidesan<assert.h>headerfileandcorrespondingassert()macrothata
programmercanusetomakeassertions.Ifanassertionfails,theassert()macroarrangestoprintadiagnosticmessage
describingtheconditionthatshouldhavebeentruebutwasnot,andthenitkillstheprogram.InC,usingassert()looks
this:
#include<assert.h>
intmyfunc(inta,doubleb)
{
assert(a<=5&&b>=17.1);
Iftheassertionfails,theprogramprintsamessagesimilartothis:
prog.c:5:assertionfailed:a<=5&&b>=17.1
TheClanguagemakesitpossibletoturntheconditionintoastringforuseinprintingthediagnosticmessage.Thisis
notpossibleinawk,sothisassert()functionalsorequiresastringversionoftheconditionthatisbeingtested.
Followingisthefunction:
#assertassertthataconditionistrue.Otherwise,exit.
functionassert(condition,string)
{
if(!condition){
printf("%s:%d:assertionfailed:%s\n",
FILENAME,FNR,string)>"/dev/stderr"
_assert_exit=1
exit1
}
}
END{
if(_assert_exit)
exit1
}
Theassert()functionteststheconditionparameter.Ifitisfalse,itprintsamessagetostandarderror,usingthestring
parametertodescribethefailedcondition.Itthensetsthevariable_assert_exittooneandexecutestheexitstatement.
TheexitstatementjumpstotheENDrule.IftheENDrulefinds_assert_exittobetrue,itexitsimmediately.
ThepurposeofthetestintheENDruleistokeepanyotherENDrulesfromrunning.Whenanassertionfails,theprogram
shouldexitimmediately.Ifnoassertionsfail,then_assert_exitisstillfalsewhentheENDruleisrunnormally,andthe
restoftheprogramsENDrulesexecute.Forallofthistoworkcorrectly,assert.awkmustbethefirstsourcefilereadby
awk.Thefunctioncanbeusedinaprograminthefollowingway:
functionmyfunc(a,b)
{
assert(a<=5&&b>=17.1,"a<=5&&b>=17.1")
}
197/479
29/7/2016
TheGNUAwkUsersGuide
Iftheassertionfails,youseeamessagesimilartothefollowing:
mydata:1357:assertionfailed:a<=5&&b>=17.1
Thereisasmallproblemwiththisversionofassert().AnENDruleisautomaticallyaddedtotheprogramcalling
assert().Normally,ifaprogramconsistsofjustaBEGINrule,theinputfilesand/orstandardinputarenotread.
However,nowthattheprogramhasanENDrule,awkattemptstoreadtheinputdatafilesorstandardinput(seeUsing
BEGIN/END),mostlikelycausingtheprogramtohangasitwaitsforinput.
Thereisasimpleworkaroundtothis:makesurethatsuchaBEGINrulealwaysendswithanexitstatement.
Next:CliffRandomFunction,Previous:AssertFunction,Up:GeneralFunctions[Contents][Index]
10.2.3RoundingNumbers
Thewayprintfandsprintf()(seePrintf)performroundingoftendependsuponthesystemsCsprintf()subroutine.
Onmanymachines,sprintf()roundingisunbiased,whichmeansitdoesntalwaysroundatrailing.5up,contraryto
naiveexpectations.Inunbiasedrounding,.5roundstoeven,ratherthanalwaysup,so1.5roundsto2but4.5roundsto
4.Thismeansthatifyouareusingaformatthatdoesrounding(e.g.,"%.0f"),youshouldcheckwhatyoursystemdoes.
Thefollowingfunctiondoestraditionalroundingitmightbeusefulifyourawksprintfdoesunbiasedrounding:
#round.awkdonormalrounding
functionround(x,ival,aval,fraction)
{
ival=int(x)#integerpart,int()truncates
#seeiffractionalpart
if(ival==x)#nofraction
returnival#ensurenodecimals
if(x<0){
aval=x#absolutevalue
ival=int(aval)
fraction=avalival
if(fraction>=.5)
returnint(x)1#2.5>3
else
returnint(x)#2.3>2
}else{
fraction=xival
if(fraction>=.5)
returnival+1
else
returnival
}
}
#testharness
#{print$0,round($0)}
Next:OrdinalFunctions,Previous:RoundFunction,Up:GeneralFunctions[Contents][Index]
10.2.4TheCliffRandomNumberGenerator
TheCliffrandomnumbergeneratorisaverysimplerandomnumbergeneratorthatpassesthenoisespheretestfor
randomnessbyshowingnostructure.Itiseasilyprogrammed,inlessthan10linesofawkcode:
#cliff_rand.awkgenerateCliffrandomnumbers
BEGIN{_cliff_seed=0.1}
198/479
29/7/2016
TheGNUAwkUsersGuide
functioncliff_rand()
{
_cliff_seed=(100*log(_cliff_seed))%1
if(_cliff_seed<0)
_cliff_seed=_cliff_seed
return_cliff_seed
}
Thisalgorithmrequiresaninitialseedof0.1.Eachnewvalueusesthecurrentseedasinputforthecalculation.Ifthe
builtinrand()function(seeNumericFunctions)isntrandomenough,youmighttryusingthisfunctioninstead.
Next:JoinFunction,Previous:CliffRandomFunction,Up:GeneralFunctions[Contents][Index]
10.2.5TranslatingBetweenCharactersandNumbers
Onecommercialimplementationofawksuppliesabuiltinfunction,ord(),whichtakesacharacterandreturnsthe
numericvalueforthatcharacterinthemachinescharacterset.Ifthestringpassedtoord()hasmorethanonecharacter,
onlythefirstoneisused.
Theinverseofthisfunctionischr()(fromthefunctionofthesamenameinPascal),whichtakesanumberandreturns
thecorrespondingcharacter.Bothfunctionsarewrittenverynicelyinawkthereisnorealreasontobuildthemintothe
awkinterpreter:
#ord.awkdoordandchr
#Globalidentifiers:
#_ord_:numericalvaluesindexedbycharacters
#_ord_init:functiontoinitialize_ord_
BEGIN{_ord_init()}
function_ord_init(low,high,i,t)
{
low=sprintf("%c",7)#BELisascii7
if(low=="\a"){#regularascii
low=0
high=127
}elseif(sprintf("%c",128+7)=="\a"){
#ascii,markparity
low=128
high=255
}else{#ebcdic(!)
low=0
high=255
}
for(i=low;i<=high;i++){
t=sprintf("%c",i)
_ord_[t]=i
}
}
Someexplanationofthenumbersusedby_ord_init()isworthwhile.Themostprominentcharactersetinusetodayis
ASCII.65Althoughan8bitbytecanhold256distinctvalues(from0to255),ASCIIonlydefinescharactersthatusethe
valuesfrom0to127.66Inthenowdistantpast,atleastoneminicomputermanufacturerusedASCII,butwithmark
parity,meaningthattheleftmostbitinthebyteisalways1.Thismeansthatonthosesystems,charactershavenumeric
valuesfrom128to255.Finally,largemainframesystemsusetheEBCDICcharacterset,whichusesall256values.
Thereareothercharactersetsinuseonsomeoldersystems,buttheyarenotreallyworthworryingabout:
functionord(str,c)
{
#onlyfirstcharacterisofinterest
199/479
29/7/2016
TheGNUAwkUsersGuide
c=substr(str,1,1)
return_ord_[c]
}
functionchr(c)
{
#forcectobenumericbyadding0
returnsprintf("%c",c+0)
}
####testcode####
#BEGIN{
#for(;;){
#printf("enteracharacter:")
#if(getlinevar<=0)
#break
#printf("ord(%s)=%d\n",var,ord(var))
#}
#}
Anobviousimprovementtothesefunctionsistomovethecodeforthe_ord_initfunctionintothebodyoftheBEGIN
rule.Itwaswrittenthiswayinitiallyforeaseofdevelopment.ThereisatestprograminaBEGINrule,totestthe
function.Itiscommentedoutforproductionuse.
Next:GetlocaltimeFunction,Previous:OrdinalFunctions,Up:GeneralFunctions[Contents][Index]
10.2.6MerginganArrayintoaString
Whendoingstringprocessing,itisoftenusefultobeabletojoinallthestringsinanarrayintoonelongstring.The
followingfunction,join(),accomplishesthistask.Itisusedlaterinseveraloftheapplicationprograms(seeSample
Programs).
Goodfunctiondesignisimportantthisfunctionneedstobegeneral,butitshouldalsohaveareasonabledefault
behavior.Itiscalledwithanarrayaswellasthebeginningandendingindicesoftheelementsinthearraytobemerged.
Thisassumesthatthearrayindicesarenumericareasonableassumption,asthearraywaslikelycreatedwithsplit()
(seeStringFunctions):
#join.awkjoinanarrayintoastring
functionjoin(array,start,end,sep,result,i)
{
if(sep=="")
sep=""
elseif(sep==SUBSEP)#magicvalue
sep=""
result=array[start]
for(i=start+1;i<=end;i++)
result=resultseparray[i]
returnresult
}
Anoptionaladditionalargumentistheseparatortousewhenjoiningthestringsbacktogether.Ifthecallersuppliesa
nonemptyvalue,join()usesitifitisnotsupplied,ithasanullvalue.Inthiscase,join()usesasinglespaceasa
defaultseparatorforthestrings.IfthevalueisequaltoSUBSEP,thenjoin()joinsthestringswithnoseparatorbetween
them.SUBSEPservesasamagicvaluetoindicatethatthereshouldbenoseparationbetweenthecomponentstrings.67
Next:ReadfileFunction,Previous:JoinFunction,Up:GeneralFunctions[Contents][Index]
10.2.7ManagingtheTimeofDay
200/479
29/7/2016
TheGNUAwkUsersGuide
Thesystime()andstrftime()functionsdescribedinTimeFunctions,providetheminimumfunctionalitynecessaryfor
dealingwiththetimeofdayinhumanreadableform.Althoughstrftime()isextensive,thecontrolformatsarenot
necessarilyeasytorememberorintuitivelyobviouswhenreadingaprogram.
Thefollowingfunction,getlocaltime(),populatesausersuppliedarraywithpreformattedtimeinformation.Itreturnsa
stringwiththecurrenttimeformattedinthesamewayasthedateutility:
#getlocaltime.awkgetthetimeofdayinausableformat
#Returnsastringintheformatofoutputofdate(1)
#Populatesthearrayargumenttimewithindividualvalues:
#time["second"]seconds(059)
#time["minute"]minutes(059)
#time["hour"]hours(023)
#time["althour"]hours(012)
#time["monthday"]dayofmonth(131)
#time["month"]monthofyear(112)
#time["monthname"]nameofthemonth
#time["shortmonth"]shortnameofthemonth
#time["year"]yearmodulo100(099)
#time["fullyear"]fullyear
#time["weekday"]dayofweek(Sunday=0)
#time["altweekday"]dayofweek(Monday=0)
#time["dayname"]nameofweekday
#time["shortdayname"]shortnameofweekday
#time["yearday"]dayofyear(0365)
#time["timezone"]abbreviationoftimezonename
#time["ampm"]AMorPMdesignation
#time["weeknum"]weeknumber,Sundayfirstday
#time["altweeknum"]weeknumber,Mondayfirstday
functiongetlocaltime(time,ret,now,i)
{
#gettimeonce,avoidsunnecessarysystemcalls
now=systime()
#returndate(1)styleoutput
ret=strftime("%a%b%e%H:%M:%S%Z%Y",now)
#clearouttargetarray
deletetime
#fillinvalues,forcenumericvaluestobe
#numericbyadding0
time["second"]=strftime("%S",now)+0
time["minute"]=strftime("%M",now)+0
time["hour"]=strftime("%H",now)+0
time["althour"]=strftime("%I",now)+0
time["monthday"]=strftime("%d",now)+0
time["month"]=strftime("%m",now)+0
time["monthname"]=strftime("%B",now)
time["shortmonth"]=strftime("%b",now)
time["year"]=strftime("%y",now)+0
time["fullyear"]=strftime("%Y",now)+0
time["weekday"]=strftime("%w",now)+0
time["altweekday"]=strftime("%u",now)+0
time["dayname"]=strftime("%A",now)
time["shortdayname"]=strftime("%a",now)
time["yearday"]=strftime("%j",now)+0
time["timezone"]=strftime("%Z",now)
time["ampm"]=strftime("%p",now)
time["weeknum"]=strftime("%U",now)+0
time["altweeknum"]=strftime("%W",now)+0
returnret
}
Thestringindicesareeasiertouseandreadthanthevariousformatsrequiredbystrftime().Thealarmprogram
201/479
29/7/2016
TheGNUAwkUsersGuide
presentedinAlarmProgram,usesthisfunction.Amoregeneraldesignforthegetlocaltime()functionwouldhave
allowedtheusertosupplyanoptionaltimestampvaluetouseinsteadofthecurrenttime.
Next:ShellQuoting,Previous:GetlocaltimeFunction,Up:GeneralFunctions[Contents][Index]
10.2.8ReadingaWholeFileatOnce
Often,itisconvenienttohavetheentirecontentsofafileavailableinmemoryasasinglestring.Astraightforwardbut
naivewaytodothatmightbeasfollows:
functionreadfile(file,tmp,contents)
{
if((getlinetmp<file)<0)
return
contents=tmp
while(getlinetmp<file)>0)
contents=contentsRTtmp
close(file)
returncontents
}
Thisfunctionreadsfromfileonerecordatatime,buildingupthefullcontentsofthefileinthelocalvariable
contents.Itworks,butisnotnecessarilyefficient.
Thefollowingfunction,basedonasuggestionbyDenisShirokov,readstheentirecontentsofthenamedfileinone
shot:
#readfile.awkreadanentirefileatonce
functionreadfile(file,tmp,save_rs)
{
save_rs=RS
RS="^$"
getlinetmp<file
close(file)
RS=save_rs
returntmp
}
ItworksbysettingRSto^$,aregularexpressionthatwillnevermatchifthefilehascontents.gawkreadsdatafromthe
fileintotmp,attemptingtomatchRS.Thematchfailsaftereachread,butfailsquickly,suchthatgawkfillstmpwiththe
entirecontentsofthefile.(SeeRecords,forinformationonRTandRS.)
Inthecasethatfileisempty,thereturnvalueisthenullstring.Thus,callingcodemayusesomethinglike:
contents=readfile("/some/path")
if(length(contents)==0)
#filewasempty
Thisteststheresulttoseeifitisemptyornot.Anequivalenttestwouldbecontents=="".
SeeExtensionSampleReadfile,foranextensionfunctionthatalsoreadsanentirefileintomemory.
Previous:ReadfileFunction,Up:GeneralFunctions[Contents][Index]
10.2.9QuotingStringstoPasstotheShell
202/479
29/7/2016
TheGNUAwkUsersGuide
MichaelBrennanoffersthefollowingprogrammingpattern,whichheusesfrequently:
#!/bin/sh
awkp='
'
input_program|awk"$awkp"|/bin/sh
Forexample,aprogramofhisnamedflacedithasthisform:
$flaceditsong="Whoope!That'sGreat"file.flac
Itgeneratesthefollowingoutput,whichistobepipedtotheshell(/bin/sh):
chmod+wfile.flac
metaflacremovetag=TITLEfile.flac
LANG=en_US.88591metaflacsettag=TITLE='Whoope!That'"'"'sGreat'file.flac
chmodwfile.flac
Notetheneedforshellquoting.Thefunctionshell_quote()doesit.SINGLEistheonecharacterstring"'"andQSINGLE
isthethreecharacterstring"\"'\"":
#shell_quotequoteanargumentforpassingtotheshell
functionshell_quote(s,#parameter
SINGLE,QSINGLE,i,X,n,ret)#locals
{
if(s=="")
return"\"\""
SINGLE="\x27"#singlequote
QSINGLE="\"\x27\""
n=split(s,X,SINGLE)
ret=SINGLEX[1]SINGLE
for(i=2;i<=n;i++)
ret=retQSINGLESINGLEX[i]SINGLE
returnret
}
Next:GetoptFunction,Previous:GeneralFunctions,Up:LibraryFunctions[Contents][Index]
10.3DatafileManagement
Thissectionpresentsfunctionsthatareusefulformanagingcommandlinedatafiles.
FiletransFunction: Afunctionforhandlingdatafiletransitions.
RewindFunction: Afunctionforrereadingthecurrentfile.
FileChecking:
Checkingthatdatafilesarereadable.
EmptyFiles:
Checkingforzerolengthfiles.
IgnoringAssigns: Treatingassignmentsasfilenames.
Next:RewindFunction,Up:DataFileManagement[Contents][Index]
10.3.1NotingDatafileBoundaries
TheBEGINandENDrulesareeachexecutedexactlyonce,atthebeginningandendofyourawkprogram,respectively(see
BEGIN/END).We(thegawkauthors)oncehadauserwhomistakenlythoughtthattheBEGINruleswereexecutedatthe
203/479
29/7/2016
TheGNUAwkUsersGuide
beginningofeachdatafileandtheENDruleswereexecutedattheendofeachdatafile.
Wheninformedthatthiswasnotthecase,theuserrequestedthatweaddnewspecialpatternstogawk,named
BEGIN_FILEandEND_FILE,thatwouldhavethedesiredbehavior.Heevensuppliedusthecodetodoso.
Addingthesespecialpatternstogawkwasntnecessarythejobcanbedonecleanlyinawkitself,asillustratedbythe
followinglibraryprogram.Itarrangestocalltwousersuppliedfunctions,beginfile()andendfile(),atthebeginning
andendofeachdatafile.Besidessolvingtheprobleminonlynine(!)linesofcode,itdoessoportablythisworkswith
anyimplementationofawk:
#transfile.awk
#
#Givetheuserahookforfilenametransitions
#
#Theusermustsupplyfunctionsbeginfile()andendfile()
#thateachtakethenameofthefilebeingstartedor
#finished,respectively.
FILENAME!=_oldfilename{
if(_oldfilename!="")
endfile(_oldfilename)
_oldfilename=FILENAME
beginfile(FILENAME)
}
END{endfile(FILENAME)}
Thisfilemustbeloadedbeforetheusersmainprogram,sothattheruleitsuppliesisexecutedfirst.
ThisrulereliesonawksFILENAMEvariable,whichautomaticallychangesforeachnewdatafile.Thecurrentfilenameis
savedinaprivatevariable,_oldfilename.IfFILENAMEdoesnotequal_oldfilename,thenanewdatafileisbeing
processedanditisnecessarytocallendfile()fortheoldfile.Becauseendfile()shouldonlybecalledifafilehasbeen
processed,theprogramfirstcheckstomakesurethat_oldfilenameisnotthenullstring.Theprogramthenassignsthe
currentfilenameto_oldfilenameandcallsbeginfile()forthefile.Because,likeallawkvariables,_oldfilenameis
initializedtothenullstring,thisruleexecutescorrectlyevenforthefirstdatafile.
TheprogramalsosuppliesanENDruletodothefinalprocessingforthelastfile.BecausethisENDrulecomesbeforeany
ENDrulessuppliedinthemainprogram,endfile()iscalledfirst.Onceagain,thevalueofmultipleBEGINandENDrules
shouldbeclear.
Ifthesamedatafileoccurstwiceinarowonthecommandline,thenendfile()andbeginfile()arenotexecutedatthe
endofthefirstpassandatthebeginningofthesecondpass.Thefollowingversionsolvestheproblem:
#ftrans.awkhandledatafiletransitions
#
#usersuppliesbeginfile()andendfile()functions
FNR==1{
if(_filename_!="")
endfile(_filename_)
_filename_=FILENAME
beginfile(FILENAME)
}
END{endfile(_filename_)}
WcProgram,showshowthislibraryfunctioncanbeusedandhowitsimplifieswritingthemainprogram.
SoWhyDoesgawkHaveBEGINFILEandENDFILE?
Youareprobablywondering,ifbeginfile()andendfile()functionscandothejob,whydoesgawkhaveBEGINFILE
andENDFILEpatterns?
204/479
29/7/2016
TheGNUAwkUsersGuide
Goodquestion.Normally,ifawkcannotopenafile,thiscausesanimmediatefatalerror.Inthiscase,thereisnoway
forauserdefinedfunctiontodealwiththeproblem,asthemechanismforcallingitreliesonthefilebeingopenandat
thefirstrecord.Thus,themainreasonforBEGINFILEistogiveyouahooktocatchfilesthatcannotbeprocessed.
ENDFILEexistsforsymmetry,andbecauseitprovidesaneasywaytodoperfilecleanupprocessing.Formore
information,refertoBEGINFILE/ENDFILE.
Next:FileChecking,Previous:FiletransFunction,Up:DataFileManagement[Contents][Index]
10.3.2RereadingtheCurrentFile
Anotherrequestforanewbuiltinfunctionwasforafunctionthatwouldmakeitpossibletorereadthecurrentfile.The
requestinguserdidntwanttohavetousegetline(seeGetline)insidealoop.
However,aslongasyouarenotintheENDrule,itisquiteeasytoarrangetoimmediatelyclosethecurrentinputfileand
thenstartoverwithitfromthetop.Forlackofabettername,wellcallthefunctionrewind():
#rewind.awkrewindthecurrentfileandstartover
functionrewind(i)
{
#shiftremainingargumentsup
for(i=ARGC;i>ARGIND;i)
ARGV[i]=ARGV[i1]
#makesuregawkknowstokeepgoing
ARGC++
#makecurrentfilenexttogetdone
ARGV[ARGIND+1]=FILENAME
#doit
nextfile
}
Therewind()functionreliesontheARGINDvariable(seeAutoset),whichisspecifictogawk.Italsoreliesonthe
nextfilekeyword(seeNextfileStatement).Becauseofthis,youshouldnotcallitfromanENDFILErule.(Thisisnt
necessaryanyway,becausegawkgoestothenextfileassoonasanENDFILErulefinishes!)
Next:EmptyFiles,Previous:RewindFunction,Up:DataFileManagement[Contents][Index]
10.3.3CheckingforReadableDatafiles
Normally,ifyougiveawkadatafilethatisntreadable,itstopswithafatalerror.Therearetimeswhenyoumightwant
tojustignoresuchfilesandkeepgoing.68Youcandothisbyprependingthefollowingprogramtoyourawkprogram:
#readable.awklibraryfiletoskipoverunreadablefiles
BEGIN{
for(i=1;i<ARGC;i++){
if(ARGV[i]~/^[azAZ_][azAZ09_]*=.*/\
||ARGV[i]==""||ARGV[i]=="/dev/stdin")
continue#assignmentorstandardinput
elseif((getlinejunk<ARGV[i])<0)#unreadable
deleteARGV[i]
else
close(ARGV[i])
}
}
Thisworks,becausethegetlinewontbefatal.RemovingtheelementfromARGVwithdeleteskipsthefile(becauseits
205/479
29/7/2016
TheGNUAwkUsersGuide
nolongerinthelist).SeealsoARGCandARGV.
BecauseawkvariablenamesonlyallowtheEnglishletters,theregularexpressioncheckpurposelydoesnotuse
characterclassessuchas[:alpha:]and[:alnum:](seeBracketExpressions).
Next:IgnoringAssigns,Previous:FileChecking,Up:DataFileManagement[Contents][Index]
10.3.4CheckingforZeroLengthFiles
Allknownawkimplementationssilentlyskipoverzerolengthfiles.Thisisabyproductofawksimplicitreadarecord
andmatchagainsttherulesloop:whenawktriestoreadarecordfromanemptyfile,itimmediatelyreceivesanendof
fileindication,closesthefile,andproceedsontothenextcommandlinedatafile,withoutexecutinganyuserlevelawk
programcode.
UsinggawksARGINDvariable(seeBuiltinVariables),itispossibletodetectwhenanemptydatafilehasbeenskipped.
SimilartothelibraryfilepresentedinFiletransFunction,thefollowinglibraryfilecallsafunctionnamedzerofile()
thattheusermustprovide.TheargumentspassedarethefilenameandthepositioninARGVwhereitwasfound:
#zerofile.awklibraryfiletoprocessemptyinputfiles
BEGIN{Argind=0}
ARGIND>Argind+1{
for(Argind++;Argind<ARGIND;Argind++)
zerofile(ARGV[Argind],Argind)
}
ARGIND!=Argind{Argind=ARGIND}
END{
if(ARGIND>Argind)
for(Argind++;Argind<=ARGIND;Argind++)
zerofile(ARGV[Argind],Argind)
}
TheuserlevelvariableArgindallowstheawkprogramtotrackitsprogressthroughARGV.Whenevertheprogramdetects
thatARGINDisgreaterthanArgind+1,itmeansthatoneormoreemptyfileswereskipped.Theactionthencalls
zerofile()foreachsuchfile,incrementingArgindalongtheway.
TheArgind!=ARGINDrulesimplykeepsArginduptodateinthenormalcase.
Finally,theENDrulecatchesthecaseofanyemptyfilesattheendofthecommandlinearguments.Notethatthetestin
theconditionoftheforloopusesthe<=operator,not<.
Previous:EmptyFiles,Up:DataFileManagement[Contents][Index]
10.3.5TreatingAssignmentsasFilenames
Occasionally,youmightnotwantawktoprocesscommandlinevariableassignments(seeAssignmentOptions).In
particular,ifyouhaveafilenamethatcontainsan=character,awktreatsthefilenameasanassignmentanddoesnot
processit.
Someusershavesuggestedanadditionalcommandlineoptionforgawktodisablecommandlineassignments.
However,somesimpleprogrammingwithalibraryfiledoesthetrick:
#noassign.awklibraryfiletoavoidtheneedfora
#specialoptionthatdisablescommandlineassignments
functiondisable_assigns(argc,argv,i)
206/479
29/7/2016
TheGNUAwkUsersGuide
{
for(i=1;i<argc;i++)
if(argv[i]~/^[azAZ_][azAZ09_]*=.*/)
argv[i]=("./"argv[i])
}
BEGIN{
if(No_command_assign)
disable_assigns(ARGC,ARGV)
}
Youthenrunyourprogramthisway:
awkvNo_command_assign=1fnoassign.awkfyourprog.awk*
Thefunctionworksbyloopingthroughthearguments.Itprepends./toanyargumentthatmatchestheformofa
variableassignment,turningthatargumentintoafilename.
TheuseofNo_command_assignallowsyoutodisablecommandlineassignmentsatinvocationtime,bygivingthe
variableatruevalue.Whennotset,itisinitiallyzero(i.e.,false),sothecommandlineargumentsareleftalone.
Next:PasswdFunctions,Previous:DataFileManagement,Up:LibraryFunctions[Contents][Index]
10.4ProcessingCommandLineOptions
MostutilitiesonPOSIXcompatiblesystemstakeoptionsonthecommandlinethatcanbeusedtochangethewaya
programbehaves.awkisanexampleofsuchaprogram(seeOptions).Often,optionstakearguments(i.e.,datathatthe
programneedstocorrectlyobeythecommandlineoption).Forexample,awksFoptionrequiresastringtouseasthe
fieldseparator.Thefirstoccurrenceonthecommandlineofeitherorastringthatdoesnotbeginwithendsthe
options.
ModernUnixsystemsprovideaCfunctionnamedgetopt()forprocessingcommandlinearguments.Theprogrammer
providesastringdescribingtheoneletteroptions.Ifanoptionrequiresanargument,itisfollowedinthestringwitha
colon.getopt()isalsopassedthecountandvaluesofthecommandlineargumentsandiscalledinaloop.getopt()
processesthecommandlineargumentsforoptionletters.Eachtimearoundtheloop,itreturnsasinglecharacter
representingthenextoptionletterthatitfinds,or?ifitfindsaninvalidoption.Whenitreturns1,therearenooptions
leftonthecommandline.
Whenusinggetopt(),optionsthatdonottakeargumentscanbegroupedtogether.Furthermore,optionsthattake
argumentsrequirethattheargumentbepresent.Theargumentcanimmediatelyfollowtheoptionletter,oritcanbea
separatecommandlineargument.
Givenahypotheticalprogramthattakesthreecommandlineoptions,a,b,andc,wherebrequiresanargument,all
ofthefollowingarevalidwaysofinvokingtheprogram:
progabfoocdata1data2data3
progacbfoodata1data2data3
progacbfoodata1data2data3
Noticethatwhentheargumentisgroupedwithitsoption,therestoftheargumentisconsideredtobetheoptions
argument.Inthisexample,acbfooindicatesthatallofthea,b,andcoptionsweresupplied,andthatfooisthe
argumenttotheboption.
getopt()providesfourexternalvariablesthattheprogrammercanuse:
optind
Theindexintheargumentvaluearray(argv)wherethefirstnonoptioncommandlineargumentcanbefound.
optarg
207/479
29/7/2016
TheGNUAwkUsersGuide
Thestringvalueoftheargumenttoanoption.
opterr
Usuallygetopt()printsanerrormessagewhenitfindsaninvalidoption.Settingopterrtozerodisablesthis
feature.(Anapplicationmightwanttoprintitsownerrormessage.)
optopt
Theletterrepresentingthecommandlineoption.
ThefollowingCfragmentshowshowgetopt()mightprocesscommandlineargumentsforawk:
int
main(intargc,char*argv[])
{
/*printourownmessage*/
opterr=0;
while((c=getopt(argc,argv,"v:f:F:W:"))!=1){
switch(c){
case'f':/*file*/
break;
case'F':/*fieldseparator*/
break;
case'v':/*variableassignment*/
break;
case'W':/*extension*/
break;
case'?':
default:
usage();
break;
}
}
Asasidepoint,gawkactuallyusestheGNUgetopt_long()functiontoprocessbothnormalandGNUstylelongoptions
(seeOptions).
Theabstractionprovidedbygetopt()isveryusefulandisquitehandyinawkprogramsaswell.Followingisanawk
versionofgetopt().Thisfunctionhighlightsoneofthegreatestweaknessesinawk,whichisthatitisverypoorat
manipulatingsinglecharacters.Repeatedcallstosubstr()arenecessaryforaccessingindividualcharacters(seeString
Functions).69
Thediscussionthatfollowswalksthroughthecodeabitatatime:
#getopt.awkDoClibrarygetopt(3)functioninawk
#Externalvariables:
#OptindindexinARGVoffirstnonoptionargument
#Optargstringvalueofargumenttocurrentoption
#Opterrifnonzero,printourowndiagnostic
#Optoptcurrentoptionletter
#Returns:
#1atendofoptions
#"?"forunrecognizedoption
#<c>acharacterrepresentingthecurrentoption
#PrivateData:
208/479
29/7/2016
TheGNUAwkUsersGuide
#_optiindexinmultiflagoption,e.g.,abc
Thefunctionstartsoutwithcommentspresentingalistoftheglobalvariablesituses,whatthereturnvaluesare,what
theymean,andanyglobalvariablesthatareprivatetothislibraryfunction.Suchdocumentationisessentialforany
program,andparticularlyforlibraryfunctions.
Thegetopt()functionfirstchecksthatitwasindeedcalledwithastringofoptions(theoptionsparameter).Ifoptions
hasazerolength,getopt()immediatelyreturns1:
functiongetopt(argc,argv,options,thisopt,i)
{
if(length(options)==0)#nooptionsgiven
return1
if(argv[Optind]==""){#alldone
Optind++
_opti=0
return1
}elseif(argv[Optind]!~/^[^:[:space:]]/){
_opti=0
return1
}
Thenextthingtocheckforistheendoftheoptions.Aendsthecommandlineoptions,asdoesanycommandline
argumentthatdoesnotbeginwitha.Optindisusedtostepthroughthearrayofcommandlineargumentsitretains
itsvalueacrosscallstogetopt(),becauseitisaglobalvariable.
Theregularexpressionthatisused,/^[^:[:space:]/,checksforafollowedbyanythingthatisnotwhitespaceand
notacolon.Ifthecurrentcommandlineargumentdoesnotmatchthispattern,itisnotanoption,anditendsoption
processing.Continuingon:
if(_opti==0)
_opti=2
thisopt=substr(argv[Optind],_opti,1)
Optopt=thisopt
i=index(options,thisopt)
if(i==0){
if(Opterr)
printf("%cinvalidoption\n",thisopt)>"/dev/stderr"
if(_opti>=length(argv[Optind])){
Optind++
_opti=0
}else
_opti++
return"?"
}
The_optivariabletracksthepositioninthecurrentcommandlineargument(argv[Optind]).Ifmultipleoptionsare
groupedtogetherwithone(e.g.,abx),itisnecessarytoreturnthemtotheuseroneatatime.
If_optiisequaltozero,itissettotwo,whichistheindexinthestringofthenextcharactertolookat(weskipthe,
whichisatpositionone).Thevariablethisoptholdsthecharacter,obtainedwithsubstr().ItissavedinOptoptforthe
mainprogramtouse.
Ifthisoptisnotintheoptionsstring,thenitisaninvalidoption.IfOpterrisnonzero,getopt()printsanerrormessage
onthestandarderrorthatissimilartothemessagefromtheCversionofgetopt().
Becausetheoptionisinvalid,itisnecessarytoskipitandmoveontothenextoptioncharacter.If_optiisgreaterthan
orequaltothelengthofthecurrentcommandlineargument,itisnecessarytomoveontothenextargument,soOptind
isincrementedand_optiisresettozero.Otherwise,Optindisleftaloneand_optiismerelyincremented.
Inanycase,becausetheoptionisinvalid,getopt()returns"?".ThemainprogramcanexamineOptoptifitneedsto
209/479
29/7/2016
TheGNUAwkUsersGuide
knowwhattheinvalidoptionletteractuallyis.Continuingon:
if(substr(options,i+1,1)==":"){
#getoptionargument
if(length(substr(argv[Optind],_opti+1))>0)
Optarg=substr(argv[Optind],_opti+1)
else
Optarg=argv[++Optind]
_opti=0
}else
Optarg=""
Iftheoptionrequiresanargument,theoptionletterisfollowedbyacolonintheoptionsstring.Ifthereareremaining
charactersinthecurrentcommandlineargument(argv[Optind]),thentherestofthatstringisassignedtoOptarg.
Otherwise,thenextcommandlineargumentisused(xFOOversusxFOO).Ineithercase,_optiisresettozero,
becausetherearenomorecharacterslefttoexamineinthecurrentcommandlineargument.Continuing:
if(_opti==0||_opti>=length(argv[Optind])){
Optind++
_opti=0
}else
_opti++
returnthisopt
}
Finally,if_optiiseitherzeroorgreaterthanthelengthofthecurrentcommandlineargument,itmeansthiselementin
argvisthroughbeingprocessed,soOptindisincrementedtopointtothenextelementinargv.Ifneitherconditionis
true,thenonly_optiisincremented,sothatthenextoptionlettercanbeprocessedonthenextcalltogetopt().
TheBEGINruleinitializesbothOpterrandOptindtoone.Opterrissettoone,becausethedefaultbehaviorisfor
getopt()toprintadiagnosticmessageuponseeinganinvalidoption.Optindissettoone,becausetheresnoreasonto
lookattheprogramname,whichisinARGV[0]:
BEGIN{
Opterr=1#defaultistodiagnose
Optind=1#skipARGV[0]
#testprogram
if(_getopt_test){
while((_go_c=getopt(ARGC,ARGV,"ab:cd"))!=1)
printf("c=<%c>,Optarg=<%s>\n",
_go_c,Optarg)
printf("nonoptionarguments:\n")
for(;Optind<ARGC;Optind++)
printf("\tARGV[%d]=<%s>\n",
Optind,ARGV[Optind])
}
}
TherestoftheBEGINruleisasimpletestprogram.Herearetheresultsoftwosamplerunsofthetestprogram:
$awkfgetopt.awkv_getopt_test=1acbARGbaxx
|c=<a>,Optarg=<>
|c=<c>,Optarg=<>
|c=<b>,Optarg=<ARG>
|nonoptionarguments:
|ARGV[3]=<bax>
|ARGV[4]=<x>
$awkfgetopt.awkv_getopt_test=1axxyzabc
|c=<a>,Optarg=<>
errorxinvalidoption
|c=<?>,Optarg=<>
|nonoptionarguments:
|ARGV[4]=<xyz>
|ARGV[5]=<abc>
210/479
29/7/2016
TheGNUAwkUsersGuide
Inbothruns,thefirstterminatestheargumentstoawk,sothatitdoesnottrytointerpretthea,etc.,asitsown
options.
NOTE:Aftergetopt()isthrough,userlevelcodemustclearoutalltheelementsofARGVfrom1toOptind,
sothatawkdoesnottrytoprocessthecommandlineoptionsasfilenames.
Using#!withtheEoptionmayhelpavoidconflictsbetweenyourprogramsoptionsandgawksoptions,asEcauses
gawktoabandonprocessingoffurtheroptions(seeExecutableScripts,andseeOptions).
SeveralofthesampleprogramspresentedinSamplePrograms,usegetopt()toprocesstheirarguments.
Next:GroupFunctions,Previous:GetoptFunction,Up:LibraryFunctions[Contents][Index]
10.5ReadingtheUserDatabase
ThePROCINFOarray(seeBuiltinVariables)providesaccesstothecurrentusersrealandeffectiveuserandgroupID
numbers,and,ifavailable,theuserssupplementarygroupset.However,becausethesearenumbers,theydonot
provideveryusefulinformationtotheaverageuser.Thereneedstobesomewaytofindtheuserinformationassociated
withtheuserandgroupIDnumbers.Thissectionpresentsasuiteoffunctionsforretrievinginformationfromtheuser
database.SeeGroupFunctions,forasimilarsuitethatretrievesinformationfromthegroupdatabase.
ThePOSIXstandarddoesnotdefinethefilewhereuserinformationiskept.Instead,itprovidesthe<pwd.h>headerfile
andseveralClanguagesubroutinesforobtaininguserinformation.Theprimaryfunctionisgetpwent(),forget
passwordentry.Thepasswordcomesfromtheoriginaluserdatabasefile,/etc/passwd,whichstoresuser
informationalongwiththeencryptedpasswords(hencethename).
Althoughanawkprogramcouldsimplyread/etc/passwddirectly,thisfilemaynotcontaincompleteinformationabout
thesystemssetofusers.70Tobesureyouareabletoproduceareadableandcompleteversionoftheuserdatabase,itis
necessarytowriteasmallCprogramthatcallsgetpwent().getpwent()isdefinedasreturningapointertoastruct
passwd.Eachtimeitiscalled,itreturnsthenextentryinthedatabase.Whentherearenomoreentries,itreturnsNULL,
thenullpointer.Whenthishappens,theCprogramshouldcallendpwent()toclosethedatabase.Followingispwcat,aC
programthatcatsthepassworddatabase:
/*
*pwcat.c
*
*Generateaprintableversionofthepassworddatabase.
*/
#include<stdio.h>
#include<pwd.h>
int
main(intargc,char**argv)
{
structpasswd*p;
while((p=getpwent())!=NULL)
printf("%s:%s:%ld:%ld:%s:%s:%s\n",
p>pw_name,p>pw_passwd,(long)p>pw_uid,
(long)p>pw_gid,p>pw_gecos,p>pw_dir,p>pw_shell);
endpwent();
return0;
}
IfyoudontunderstandC,dontworryaboutit.Theoutputfrompwcatistheuserdatabase,inthetraditional
/etc/passwdformatofcolonseparatedfields.Thefieldsare:
Loginname
Theusersloginname.
211/479
29/7/2016
TheGNUAwkUsersGuide
Encryptedpassword
Theusersencryptedpassword.Thismaynotbeavailableonsomesystems.
UserID
TheusersnumericuserIDnumber.(Onsomesystems,itsaClong,andnotanint.Thus,wecastittolongfor
allcases.)
GroupID
TheusersnumericgroupIDnumber.(Similarcommentsaboutlongversusintapplyhere.)
Fullname
Theusersfullname,andperhapsotherinformationassociatedwiththeuser.
Homedirectory
Theuserslogin(orhome)directory(familiartoshellprogrammersas$HOME).
Loginshell
Theprogramthatisrunwhentheuserlogsin.Thisisusuallyashell,suchasBash.
Afewlinesrepresentativeofpwcatsoutputareasfollows:
$pwcat
|root:x:0:1:Operator:/:/bin/sh
|nobody:*:65534:65534::/:
|daemon:*:1:1::/:
|sys:*:2:2::/:/bin/csh
|bin:*:3:3::/bin:
|arnold:xyzzy:2076:10:ArnoldRobbins:/home/arnold:/bin/sh
|miriam:yxaay:112:10:MiriamRobbins:/home/miriam:/bin/sh
|andy:abcca2:113:10:AndyJacobs:/home/andy:/bin/sh
Withthatintroduction,followingisagroupoffunctionsforgettinguserinformation.Thereareseveralfunctionshere,
correspondingtotheCfunctionsofthesamenames:
#passwd.awkaccesspasswordfileinformation
BEGIN{
#tailorthistosuityoursystem
_pw_awklib="/usr/local/libexec/awk/"
}
function_pw_init(oldfs,oldrs,olddol0,pwcat,using_fw,using_fpat)
{
if(_pw_inited)
return
oldfs=FS
oldrs=RS
olddol0=$0
using_fw=(PROCINFO["FS"]=="FIELDWIDTHS")
using_fpat=(PROCINFO["FS"]=="FPAT")
FS=":"
RS="\n"
pwcat=_pw_awklib"pwcat"
while((pwcat|getline)>0){
_pw_byname[$1]=$0
_pw_byuid[$3]=$0
212/479
29/7/2016
TheGNUAwkUsersGuide
_pw_bycount[++_pw_total]=$0
}
close(pwcat)
_pw_count=0
_pw_inited=1
FS=oldfs
if(using_fw)
FIELDWIDTHS=FIELDWIDTHS
elseif(using_fpat)
FPAT=FPAT
RS=oldrs
$0=olddol0
}
TheBEGINrulesetsaprivatevariabletothedirectorywherepwcatisstored.Becauseitisusedtohelpoutanawklibrary
routine,wehavechosentoputitin/usr/local/libexec/awkhowever,youmightwantittobeinadifferentdirectory
onyoursystem.
Thefunction_pw_init()fillsthreecopiesoftheuserinformationintothreeassociativearrays.Thearraysareindexed
byusername(_pw_byname),byuserIDnumber(_pw_byuid),andbyorderofoccurrence(_pw_bycount).Thevariable
_pw_initedisusedforefficiency,as_pw_init()needstobecalledonlyonce.
Becausethisfunctionusesgetlinetoreadinformationfrompwcat,itfirstsavesthevaluesofFS,RS,and$0.Itnotesin
thevariableusing_fwwhetherfieldsplittingwithFIELDWIDTHSisineffectornot.Doingsoisnecessary,asthese
functionscouldbecalledfromanywherewithinausersprogram,andtheusermayhavehisorherownwayofsplitting
recordsandfields.Thismakesitpossibletorestorethecorrectfieldsplittingmechanismlater.Thetestcanonlybetrue
forgawk.ItisfalseifusingFSorFPAT,oronsomeotherawkimplementation.
ThecodethatchecksforusingFPAT,usingusing_fpatandPROCINFO["FS"],issimilar.
Themainpartofthefunctionusesalooptoreaddatabaselines,splitthelinesintofields,andthenstorethelinesinto
eacharrayasnecessary.Whentheloopisdone,_pw_init()cleansupbyclosingthepipeline,setting_pw_initedtoone,
andrestoringFS(andFIELDWIDTHSorFPATifnecessary),RS,and$0.Theuseof_pw_countisexplainedshortly.
Thegetpwnam()functiontakesausernameasastringargument.Ifthatuserisinthedatabase,itreturnstheappropriate
line.Otherwise,itreliesonthearrayreferencetoanonexistentelementtocreatetheelementwiththenullstringasits
value:
functiongetpwnam(name)
{
_pw_init()
return_pw_byname[name]
}
Similarly,thegetpwuid()functiontakesauserIDnumberargument.Ifthatusernumberisinthedatabase,itreturnsthe
appropriateline.Otherwise,itreturnsthenullstring:
functiongetpwuid(uid)
{
_pw_init()
return_pw_byuid[uid]
}
Thegetpwent()functionsimplystepsthroughthedatabase,oneentryatatime.Ituses_pw_counttotrackitscurrent
positioninthe_pw_bycountarray:
functiongetpwent()
{
_pw_init()
if(_pw_count<_pw_total)
return_pw_bycount[++_pw_count]
return""
}
213/479
29/7/2016
TheGNUAwkUsersGuide
Theendpwent()functionresets_pw_counttozero,sothatsubsequentcallstogetpwent()startoveragain:
functionendpwent()
{
_pw_count=0
}
Aconsciousdesigndecisioninthissuiteisthateachsubroutinecalls_pw_init()toinitializethedatabasearrays.The
overheadofrunningaseparateprocesstogeneratetheuserdatabase,andtheI/Otoscanit,areonlyincurredifthe
usersmainprogramactuallycallsoneofthesefunctions.Ifthislibraryfileisloadedalongwithausersprogram,but
noneoftheroutinesareevercalled,thenthereisnoextraruntimeoverhead.(Thealternativeismovethebodyof
_pw_init()intoaBEGINrule,whichalwaysrunspwcat.Thissimplifiesthecodebutrunsanextraprocessthatmaynever
beneeded.)
Inturn,calling_pw_init()isnottooexpensive,becausethe_pw_initedvariablekeepstheprogramfromreadingthe
datamorethanonce.Ifyouareworriedaboutsqueezingeverylastcycleoutofyourawkprogram,thecheckof
_pw_initedcouldbemovedoutof_pw_init()andduplicatedinalltheotherfunctions.Inpractice,thisisnotnecessary,
asmostawkprogramsareI/Obound,andsuchachangewouldclutterupthecode.
TheidprograminIdProgram,usesthesefunctions.
Next:WalkingArrays,Previous:PasswdFunctions,Up:LibraryFunctions[Contents][Index]
10.6ReadingtheGroupDatabase
MuchofthediscussionpresentedinPasswdFunctions,appliestothegroupdatabaseaswell.Althoughtherehas
traditionallybeenawellknownfile(/etc/group)inawellknownformat,thePOSIXstandardonlyprovidesasetofC
libraryroutines(<grp.h>andgetgrent())foraccessingtheinformation.Eventhoughthisfilemayexist,itmaynothave
completeinformation.Therefore,aswiththeuserdatabase,itisnecessarytohaveasmallCprogramthatgeneratesthe
groupdatabaseasitsoutput.grcat,aCprogramthatcatsthegroupdatabase,isasfollows:
/*
*grcat.c
*
*Generateaprintableversionofthegroupdatabase.
*/
#include<stdio.h>
#include<grp.h>
int
main(intargc,char**argv)
{
structgroup*g;
inti;
while((g=getgrent())!=NULL){
printf("%s:%s:%ld:",g>gr_name,g>gr_passwd,
(long)g>gr_gid);
for(i=0;g>gr_mem[i]!=NULL;i++){
printf("%s",g>gr_mem[i]);
if(g>gr_mem[i+1]!=NULL)
putchar(',');
}
putchar('\n');
}
endgrent();
return0;
}
Eachlineinthegroupdatabaserepresentsonegroup.Thefieldsareseparatedwithcolonsandrepresentthefollowing
information:
214/479
29/7/2016
TheGNUAwkUsersGuide
GroupName
Thegroupsname.
GroupPassword
Thegroupsencryptedpassword.Inpractice,thisfieldisneveruseditisusuallyemptyorsetto*.
GroupIDNumber
ThegroupsnumericgroupIDnumbertheassociationofnametonumbermustbeuniquewithinthefile.(On
somesystemsitsaClong,andnotanint.Thus,wecastittolongforallcases.)
GroupMemberList
Acommaseparatedlistofusernames.Theseusersaremembersofthegroup.ModernUnixsystemsallowusers
tobemembersofseveralgroupssimultaneously.Ifyoursystemdoes,thenthereareelements"group1"through
"groupN"inPROCINFOforthosegroupIDnumbers.(NotethatPROCINFOisagawkextensionseeBuiltinVariables.)
Hereiswhatrunninggrcatmightproduce:
$grcat
|wheel:*:0:arnold
|nogroup:*:65534:
|daemon:*:1:
|kmem:*:2:
|staff:*:10:arnold,miriam,andy
|other:*:20:
Herearethefunctionsforobtaininginformationfromthegroupdatabase.Thereareseveral,modeledaftertheClibrary
functionsofthesamenames:
#group.awkfunctionsfordealingwiththegroupfile
BEGIN{
#Changetosuityoursystem
_gr_awklib="/usr/local/libexec/awk/"
}
function_gr_init(oldfs,oldrs,olddol0,grcat,
using_fw,using_fpat,n,a,i)
{
if(_gr_inited)
return
oldfs=FS
oldrs=RS
olddol0=$0
using_fw=(PROCINFO["FS"]=="FIELDWIDTHS")
using_fpat=(PROCINFO["FS"]=="FPAT")
FS=":"
RS="\n"
grcat=_gr_awklib"grcat"
while((grcat|getline)>0){
if($1in_gr_byname)
_gr_byname[$1]=_gr_byname[$1]","$4
else
_gr_byname[$1]=$0
if($3in_gr_bygid)
_gr_bygid[$3]=_gr_bygid[$3]","$4
else
_gr_bygid[$3]=$0
n=split($4,a,"[\t]*,[\t]*")
215/479
29/7/2016
TheGNUAwkUsersGuide
for(i=1;i<=n;i++)
if(a[i]in_gr_groupsbyuser)
_gr_groupsbyuser[a[i]]=gr_groupsbyuser[a[i]]""$1
else
_gr_groupsbyuser[a[i]]=$1
_gr_bycount[++_gr_count]=$0
}
close(grcat)
_gr_count=0
_gr_inited++
FS=oldfs
if(using_fw)
FIELDWIDTHS=FIELDWIDTHS
elseif(using_fpat)
FPAT=FPAT
RS=oldrs
$0=olddol0
}
TheBEGINrulesetsaprivatevariabletothedirectorywheregrcatisstored.Becauseitisusedtohelpoutanawklibrary
routine,wehavechosentoputitin/usr/local/libexec/awk.Youmightwantittobeinadifferentdirectoryonyour
system.
Theseroutinesfollowthesamegeneraloutlineastheuserdatabaseroutines(seePasswdFunctions).The_gr_inited
variableisusedtoensurethatthedatabaseisscannednomorethanonce.The_gr_init()functionfirstsavesFS,RS,and
$0,andthensetsFSandRStothecorrectvaluesforscanningthegroupinformation.Italsotakescaretonotewhether
FIELDWIDTHSorFPATisbeingused,andtorestoretheappropriatefieldsplittingmechanism.
Thegroupinformationisstoredinseveralassociativearrays.Thearraysareindexedbygroupname(_gr_byname),by
groupIDnumber(_gr_bygid),andbypositioninthedatabase(_gr_bycount).Thereisanadditionalarrayindexedby
username(_gr_groupsbyuser),whichisaspaceseparatedlistofgroupstowhicheachuserbelongs.
Unlikeintheuserdatabase,itispossibletohavemultiplerecordsinthedatabaseforthesamegroup.Thisiscommon
whenagrouphasalargenumberofmembers.Apairofsuchentriesmightlooklikethefollowing:
tvpeople:*:101:johnny,jay,arsenio
tvpeople:*:101:david,conan,tom,joan
Forthisreason,_gr_init()lookstoseeifagroupnameorgroupIDnumberisalreadyseen.Ifso,theusernamesare
simplyconcatenatedontothepreviouslistofusers.71
Finally,_gr_init()closesthepipelinetogrcat,restoresFS(andFIELDWIDTHSorFPAT,ifnecessary),RS,and$0,
initializes_gr_counttozero(itisusedlater),andmakes_gr_initednonzero.
Thegetgrnam()functiontakesagroupnameasitsargument,andifthatgroupexists,itisreturned.Otherwise,itrelies
onthearrayreferencetoanonexistentelementtocreatetheelementwiththenullstringasitsvalue:
functiongetgrnam(group)
{
_gr_init()
return_gr_byname[group]
}
Thegetgrgid()functionissimilarittakesanumericgroupIDandlooksuptheinformationassociatedwiththatgroup
ID:
functiongetgrgid(gid)
{
_gr_init()
return_gr_bygid[gid]
}
Thegetgruser()functiondoesnothaveaCcounterpart.Ittakesausernameandreturnsthelistofgroupsthathavethe
216/479
29/7/2016
TheGNUAwkUsersGuide
userasamember:
functiongetgruser(user)
{
_gr_init()
return_gr_groupsbyuser[user]
}
Thegetgrent()functionstepsthroughthedatabaseoneentryatatime.Ituses_gr_counttotrackitspositioninthelist:
functiongetgrent()
{
_gr_init()
if(++_gr_countin_gr_bycount)
return_gr_bycount[_gr_count]
return""
}
Theendgrent()functionresets_gr_counttozerosothatgetgrent()canstartoveragain:
functionendgrent()
{
_gr_count=0
}
Aswiththeuserdatabaseroutines,eachfunctioncalls_gr_init()toinitializethearrays.Doingsoonlyincurstheextra
overheadofrunninggrcatifthesefunctionsareused(asopposedtomovingthebodyof_gr_init()intoaBEGINrule).
Mostoftheworkisinscanningthedatabaseandbuildingthevariousassociativearrays.Thefunctionsthattheuser
callsarethemselvesverysimple,relyingonawksassociativearraystodowork.
TheidprograminIdProgram,usesthesefunctions.
Next:LibraryFunctionsSummary,Previous:GroupFunctions,Up:LibraryFunctions[Contents][Index]
10.7TraversingArraysofArrays
ArraysofArrays,describedhowgawkprovidesarraysofarrays.Inparticular,anyelementofanarraymaybeeithera
scalaroranotherarray.Theisarray()function(seeTypeFunctions)letsyoudistinguishanarrayfromascalar.The
followingfunction,walk_array(),recursivelytraversesanarray,printingtheelementindicesandvalues.Youcallit
withthearrayandastringrepresentingthenameofthearray:
functionwalk_array(arr,name,i)
{
for(iinarr){
if(isarray(arr[i]))
walk_array(arr[i],(name"["i"]"))
else
printf("%s[%s]=%s\n",name,i,arr[i])
}
}
Itworksbyloopingovereachelementofthearray.Ifanygivenelementisitselfanarray,thefunctioncallsitself
recursively,passingthesubarrayandanewstringrepresentingthecurrentindex.Otherwise,thefunctionsimplyprints
theelementsname,index,andvalue.Hereisamainprogramtodemonstrate:
BEGIN{
a[1]=1
a[2][1]=21
a[2][2]=22
a[3]=3
a[4][1][1]=411
a[4][2]=42
217/479
29/7/2016
TheGNUAwkUsersGuide
walk_array(a,"a")
}
Whenrun,theprogramproducesthefollowingoutput:
$gawkfwalk_array.awk
|a[1]=1
|a[2][1]=21
|a[2][2]=22
|a[3]=3
|a[4][1][1]=411
|a[4][2]=42
Thefunctionjustpresentedsimplyprintsthenameandvalueofeachscalararrayelement.However,itiseasyto
generalizeit,bypassinginthenameofafunctiontocallwhenwalkinganarray.Themodifiedfunctionlookslikethis:
functionprocess_array(arr,name,process,do_arrays,i,new_name)
{
for(iinarr){
new_name=(name"["i"]")
if(isarray(arr[i])){
if(do_arrays)
@process(new_name,arr[i])
process_array(arr[i],new_name,process,do_arrays)
}else
@process(new_name,arr[i])
}
}
Theargumentsareasfollows:
arr
Thearray.
name
Thenameofthearray(astring).
process
Thenameofthefunctiontocall.
do_arrays
Ifthisistrue,thefunctioncanhandleelementsthataresubarrays.
Ifsubarraysaretobeprocessed,thatisdonebeforewalkingthemfurther.
Whenrunwiththefollowingscaffolding,thefunctionproducesthesameresultsasdoestheearlierversionof
walk_array():
BEGIN{
a[1]=1
a[2][1]=21
a[2][2]=22
a[3]=3
a[4][1][1]=411
a[4][2]=42
process_array(a,"a","do_print",0)
}
functiondo_print(name,element)
{
218/479
29/7/2016
TheGNUAwkUsersGuide
printf"%s=%s\n",name,element
}
Next:LibraryExercises,Previous:WalkingArrays,Up:LibraryFunctions[Contents][Index]
10.8Summary
ReadingprogramsisanexcellentwaytolearnGoodProgramming.Thefunctionsandprogramsprovidedinthis
chapterandthenextareintendedtoservethatpurpose.
Whenwritinggeneralpurposelibraryfunctions,putsomethoughtintohowtonameanyglobalvariablessothat
theywontconflictwithvariablesfromausersprogram.
Thefunctionspresentedherefitintothefollowingcategories:
Generalproblems
Numbertostringconversion,testingassertions,rounding,randomnumbergeneration,converting
characterstonumbers,joiningstrings,gettingeasilyusabletimeofdayinformation,andreadingawhole
fileinoneshot
Managingdatafiles
Notingdatafileboundaries,rereadingthecurrentfile,checkingforreadablefiles,checkingforzerolength
files,andtreatingassignmentsasfilenames
Processingcommandlineoptions
AnawkversionofthestandardCgetopt()function
Readingtheuserandgroupdatabases
TwosetsofroutinesthatparalleltheClibraryversions
Traversingarraysofarrays
Twofunctionsthattraverseanarrayofarraystoanydepth
Previous:LibraryFunctionsSummary,Up:LibraryFunctions[Contents][Index]
10.9Exercises
1.InEmptyFiles,wepresentedthezerofile.awkprogram,whichmadeuseofgawksARGINDvariable.Canthis
problembesolvedwithoutrelyingonARGIND?Ifso,how?
2.Asarelatedchallenge,revisethatcodetohandlethecasewhereaninterveningvalueinARGVisavariable
assignment.
3.WalkingArrays,presentedafunctionthatwalkedamultidimensionalarraytoprintitout.However,walkingan
arrayandprocessingeachelementisageneralpurposeoperation.Generalizethewalk_array()functionby
addinganadditionalparameternamedprocess.
Then,insidetheloop,insteadofprintingthearrayelementsindexandvalue,usetheindirectfunctioncallsyntax
(seeIndirectCalls)onprocess,passingittheindexandthevalue.
Whencallingwalk_array(),youwouldpassthenameofauserdefinedfunctionthatexpectstoreceiveanindex
andavalue,andthenprocessestheelement.
Testyournewversionbyprintingthearrayyoushouldendupwithoutputidenticaltothatoftheoriginal
version.
219/479
29/7/2016
TheGNUAwkUsersGuide
Next:AdvancedFeatures,Previous:LibraryFunctions,Up:Top[Contents][Index]
LibraryFunctions,presentstheideathatreadingprogramsinalanguagecontributestolearningthatlanguage.This
chaptercontinuesthattheme,presentingapotpourriofawkprogramsforyourreadingenjoyment.Therearethree
sections.Thefirstdescribeshowtoruntheprogramspresentedinthischapter.
ThesecondpresentsawkversionsofseveralcommonPOSIXutilities.Theseareprogramsthatyouarehopefullyalready
familiarwith,andthereforewhoseproblemsareunderstood.Byreimplementingtheseprogramsinawk,youcanfocus
ontheawkrelatedaspectsofsolvingtheprogrammingproblems.
Thethirdisagrabbagofinterestingprograms.Thesesolveanumberofdifferentdatamanipulationandmanagement
problems.Manyoftheprogramsareshort,whichemphasizesawksabilitytodoalotinjustafewlinesofcode.
ManyoftheseprogramsuselibraryfunctionspresentedinLibraryFunctions.
RunningExamples:
Howtoruntheseexamples.
Clones:
Clonesofcommonutilities.
MiscellaneousPrograms: Someinterestingawkprograms.
ProgramsSummary:
Summaryofprograms.
ProgramsExercises:
Exercises.
Next:Clones,Up:SamplePrograms[Contents][Index]
11.1RunningtheExamplePrograms
Torunagivenprogram,youwouldtypicallydosomethinglikethis:
awkfprogramoptionsfiles
Here,programisthenameoftheawkprogram(suchascut.awk),optionsareanycommandlineoptionsfortheprogram
thatstartwitha,andfilesaretheactualdatafiles.
Ifyoursystemsupportsthe#!executableinterpretermechanism(seeExecutableScripts),youcaninsteadrunyour
programdirectly:
cut.awkc18myfiles>results
Ifyourawkisnotgawk,youmayinsteadneedtousethis:
cut.awkc18myfiles>results
Next:MiscellaneousPrograms,Previous:RunningExamples,Up:SamplePrograms[Contents][Index]
11.2ReinventingWheelsforFunandProfit
ThissectionpresentsanumberofPOSIXutilitiesimplementedinawk.Reinventingtheseprogramsinawkisoften
enjoyable,becausethealgorithmscanbeveryclearlyexpressed,andthecodeisusuallyveryconciseandsimple.This
istruebecauseawkdoessomuchforyou.
Itshouldbenotedthattheseprogramsarenotnecessarilyintendedtoreplacetheinstalledversionsonyoursystem.Nor
mayalloftheseprogramsbefullycompliantwiththemostrecentPOSIXstandard.Thisisnotaproblemtheirpurpose
istoillustrateawklanguageprogrammingforrealworldtasks.
Theprogramsarepresentedinalphabeticalorder.
220/479
29/7/2016
TheGNUAwkUsersGuide
CutProgram: Thecututility.
EgrepProgram: Theegreputility.
IdProgram:
Theidutility.
SplitProgram: Thesplitutility.
TeeProgram: Theteeutility.
UniqProgram: Theuniqutility.
WcProgram: Thewcutility.
Next:EgrepProgram,Up:Clones[Contents][Index]
11.2.1CuttingOutFieldsandColumns
Thecututilityselects,orcuts,charactersorfieldsfromitsstandardinputandsendsthemtoitsstandardoutput.Fields
areseparatedbyTABsbydefault,butyoumaysupplyacommandlineoptiontochangethefielddelimiter(i.e.,the
fieldseparatorcharacter).cutsdefinitionoffieldsislessgeneralthanawks.
Acommonuseofcutmightbetopulloutjusttheloginnamesofloggedonusersfromtheoutputofwho.Forexample,
thefollowingpipelinegeneratesasorted,uniquelistoftheloggedonusers:
who|cutc18|sort|uniq
Theoptionsforcutare:
clist
Uselistasthelistofcharacterstocutout.Itemswithinthelistmaybeseparatedbycommas,andrangesof
characterscanbeseparatedwithdashes.Thelist18,15,2235specifiescharacters1through8,15,and22
through35.
flist
Uselistasthelistoffieldstocutout.
ddelim
UsedelimasthefieldseparatorcharacterinsteadoftheTABcharacter.
s
Suppressprintingoflinesthatdonotcontainthefielddelimiter.
Theawkimplementationofcutusesthegetopt()libraryfunction(seeGetoptFunction)andthejoin()libraryfunction
(seeJoinFunction).
Theprogrambeginswithacommentdescribingtheoptions,thelibraryfunctionsneeded,andausage()functionthat
printsoutausagemessageandexits.usage()iscalledifinvalidargumentsaresupplied:
#cut.awkimplementcutinawk
#Options:
#flistCutfields
#dcFielddelimitercharacter
#clistCutcharacters
#
#sSuppresslineswithoutthedelimiter
#
#Requiresgetopt()andjoin()libraryfunctions
functionusage()
{
221/479
29/7/2016
TheGNUAwkUsersGuide
print("usage:cut[flist][dc][s][files...]")>"/dev/stderr"
print("usage:cut[clist][files...]")>"/dev/stderr"
exit1
}
NextcomesaBEGINrulethatparsesthecommandlineoptions.ItsetsFStoasingleTABcharacter,becausethatiscuts
defaultfieldseparator.Therulethensetstheoutputfieldseparatortobethesameastheinputfieldseparator.Aloop
usinggetopt()stepsthroughthecommandlineoptions.Exactlyoneofthevariablesby_fieldsorby_charsissetto
true,toindicatethatprocessingshouldbedonebyfieldsorbycharacters,respectively.Whencuttingbycharacters,the
outputfieldseparatorissettothenullstring:
BEGIN{
FS="\t"#default
OFS=FS
while((c=getopt(ARGC,ARGV,"sf:c:d:"))!=1){
if(c=="f"){
by_fields=1
fieldlist=Optarg
}elseif(c=="c"){
by_chars=1
fieldlist=Optarg
OFS=""
}elseif(c=="d"){
if(length(Optarg)>1){
printf("cut:usingfirstcharacterof%s"\
"fordelimiter\n",Optarg)>"/dev/stderr"
Optarg=substr(Optarg,1,1)
}
FS=Optarg
OFS=FS
if(FS=="")#defeatawksemantics
FS="[]"
}elseif(c=="s")
suppress=1
else
usage()
}
#Clearoutoptions
for(i=1;i<Optind;i++)
ARGV[i]=""
Thecodemusttakespecialcarewhenthefielddelimiterisaspace.Usingasinglespace("")forthevalueofFSis
incorrectawkwouldseparatefieldswithrunsofspaces,TABs,and/ornewlines,andwewantthemtobeseparated
withindividualspaces.Alsorememberthataftergetopt()isthrough(asdescribedinGetoptFunction),wehavetoclear
outalltheelementsofARGVfrom1toOptind,sothatawkdoesnottrytoprocessthecommandlineoptionsasfilenames.
Afterdealingwiththecommandlineoptions,theprogramverifiesthattheoptionsmakesense.Onlyoneortheotherof
candfshouldbeused,andbothrequireafieldlist.Thentheprogramcallseitherset_fieldlist()orset_charlist()
topullapartthelistoffieldsorcharacters:
if(by_fields&&by_chars)
usage()
if(by_fields==0&&by_chars==0)
by_fields=1#default
if(fieldlist==""){
print"cut:needslistforcorf">"/dev/stderr"
exit1
}
if(by_fields)
set_fieldlist()
else
set_charlist()
222/479
29/7/2016
TheGNUAwkUsersGuide
}
set_fieldlist()splitsthefieldlistapartatthecommasintoanarray.Then,foreachelementofthearray,itlookstosee
iftheelementisactuallyarange,andifso,splitsitapart.Thefunctioncheckstherangetomakesurethatthefirst
numberissmallerthanthesecond.Eachnumberinthelistisaddedtotheflistarray,whichsimplyliststhefieldsthat
willbeprinted.Normalfieldsplittingisused.Theprogramletsawkhandlethejobofdoingthefieldsplitting:
functionset_fieldlist(n,m,i,j,k,f,g)
{
n=split(fieldlist,f,",")
j=1#indexinflist
for(i=1;i<=n;i++){
if(index(f[i],"")!=0){#arange
m=split(f[i],g,"")
if(m!=2||g[1]>=g[2]){
printf("cut:badfieldlist:%s\n",
f[i])>"/dev/stderr"
exit1
}
for(k=g[1];k<=g[2];k++)
flist[j++]=k
}else
flist[j++]=f[i]
}
nfields=j1
}
Theset_charlist()functionismorecomplicatedthanset_fieldlist().TheideahereistousegawksFIELDWIDTHS
variable(seeConstantSize),whichdescribesconstantwidthinput.Whenusingacharacterlist,thatisexactlywhatwe
have.
SettingupFIELDWIDTHSismorecomplicatedthansimplylistingthefieldsthatneedtobeprinted.Wehavetokeeptrack
ofthefieldstoprintandalsotheinterveningcharactersthathavetobeskipped.Forexample,supposeyouwanted
characters1through8,15,and22through35.Youwouldusec18,15,2235.ThenecessaryvalueforFIELDWIDTHS
is"861614".Thisyieldsfivefields,andthefieldstoprintare$1,$3,and$5.Theintermediatefieldsarefiller,which
isstuffinbetweenthedesireddata.flistliststhefieldstoprint,andttracksthecompletefieldlist,includingfiller
fields:
functionset_charlist(field,i,j,f,g,n,m,t,
filler,last,len)
{
field=1#counttotalfields
n=split(fieldlist,f,",")
j=1#indexinflist
for(i=1;i<=n;i++){
if(index(f[i],"")!=0){#range
m=split(f[i],g,"")
if(m!=2||g[1]>=g[2]){
printf("cut:badcharacterlist:%s\n",
f[i])>"/dev/stderr"
exit1
}
len=g[2]g[1]+1
if(g[1]>1)#computelengthoffiller
filler=g[1]last1
else
filler=0
if(filler)
t[field++]=filler
t[field++]=len#lengthoffield
last=g[2]
flist[j++]=field1
}else{
223/479
29/7/2016
TheGNUAwkUsersGuide
if(f[i]>1)
filler=f[i]last1
else
filler=0
if(filler)
t[field++]=filler
t[field++]=1
last=f[i]
flist[j++]=field1
}
}
FIELDWIDTHS=join(t,1,field1)
nfields=j1
}
Nextistherulethatprocessesthedata.Ifthesoptionisgiven,thensuppressistrue.Thefirstifstatementmakessure
thattheinputrecorddoeshavethefieldseparator.Ifcutisprocessingfields,suppressistrue,andthefieldseparator
characterisnotintherecord,thentherecordisskipped.
Iftherecordisvalid,thengawkhassplitthedataintofields,eitherusingthecharacterinFSorusingfixedlengthfields
andFIELDWIDTHS.Theloopgoesthroughthelistoffieldsthatshouldbeprinted.Thecorrespondingfieldisprintedifit
containsdata.Ifthenextfieldalsohasdata,thentheseparatorcharacteriswrittenoutbetweenthefields:
{
if(by_fields&&suppress&&index($0,FS)==0)
next
for(i=1;i<=nfields;i++){
if($flist[i]!=""){
printf"%s",$flist[i]
if(i<nfields&&$flist[i+1]!="")
printf"%s",OFS
}
}
print""
}
ThisversionofcutreliesongawksFIELDWIDTHSvariabletodothecharacterbasedcutting.Itispossibleinotherawk
implementationstousesubstr()(seeStringFunctions),butitisalsoextremelypainful.TheFIELDWIDTHSvariable
suppliesanelegantsolutiontotheproblemofpickingtheinputlineapartbycharacters.
Next:IdProgram,Previous:CutProgram,Up:Clones[Contents][Index]
11.2.2SearchingforRegularExpressionsinFiles
Theegreputilitysearchesfilesforpatterns.Itusesregularexpressionsthatarealmostidenticaltothoseavailableinawk
(seeRegexp).Youinvokeitasfollows:
egrep[options]'pattern'files
Thepatternisaregularexpression.Intypicalusage,theregularexpressionisquotedtopreventtheshellfrom
expandinganyofthespecialcharactersasfilenamewildcards.Normally,egrepprintsthelinesthatmatched.Ifmultiple
filenamesareprovidedonthecommandline,eachoutputlineisprecededbythenameofthefileandacolon.
Theoptionstoegrepareasfollows:
c
Printoutacountofthelinesthatmatchedthepattern,insteadofthelinesthemselves.
s
224/479
29/7/2016
TheGNUAwkUsersGuide
Besilent.Nooutputisproducedandtheexitvalueindicateswhetherthepatternwasmatched.
v
Invertthesenseofthetest.egrepprintsthelinesthatdonotmatchthepatternandexitssuccessfullyifthepattern
isnotmatched.
i
Ignorecasedistinctionsinboththepatternandtheinputdata.
l
Onlyprint(list)thenamesofthefilesthatmatched,notthelinesthatmatched.
epattern
Usepatternastheregexptomatch.Thepurposeoftheeoptionistoallowpatternsthatstartwitha.
Thisversionusesthegetopt()libraryfunction(seeGetoptFunction)andthefiletransitionlibraryprogram(see
FiletransFunction).
TheprogrambeginswithadescriptivecommentandthenaBEGINrulethatprocessesthecommandlineargumentswith
getopt().Thei(ignorecase)optionisparticularlyeasywithgawkwejustusetheIGNORECASEpredefinedvariable(see
BuiltinVariables):
#egrep.awksimulateegrepinawk
#
#Options:
#ccountoflines
#ssilentuseexitvalue
#vinverttest,successifnomatch
#iignorecase
#lprintfilenamesonly
#eargumentispattern
#
#Requiresgetoptandfiletransitionlibraryfunctions
BEGIN{
while((c=getopt(ARGC,ARGV,"ce:svil"))!=1){
if(c=="c")
count_only++
elseif(c=="s")
no_print++
elseif(c=="v")
invert++
elseif(c=="i")
IGNORECASE=1
elseif(c=="l")
filenames_only++
elseif(c=="e")
pattern=Optarg
else
usage()
}
Nextcomesthecodethathandlestheegrepspecificbehavior.Ifnopatternissuppliedwithe,thefirstnonoptionon
thecommandlineisused.TheawkcommandlineargumentsuptoARGV[Optind]arecleared,sothatawkwonttryto
processthemasfiles.Ifnofilesarespecified,thestandardinputisused,andifmultiplefilesarespecified,wemake
suretonotethissothatthefilenamescanprecedethematchedlinesintheoutput:
if(pattern=="")
pattern=ARGV[Optind++]
225/479
29/7/2016
TheGNUAwkUsersGuide
ARGV[i]=""
if(Optind>=ARGC){
ARGV[1]=""
ARGC=2
}elseif(ARGCOptind>1)
do_filenames++
#if(IGNORECASE)
#pattern=tolower(pattern)
}
Thelasttwolinesarecommentedout,astheyarenotneededingawk.Theyshouldbeuncommentedifyouhavetouse
anotherversionofawk.
Thenextsetoflinesshouldbeuncommentedifyouarenotusinggawk.Thisruletranslatesallthecharactersintheinput
lineintolowercaseiftheioptionisspecified.72Theruleiscommentedoutasitisnotnecessarywithgawk:
#{
#if(IGNORECASE)
#$0=tolower($0)
#}
Thebeginfile()functioniscalledbytheruleinftrans.awkwheneachnewfileisprocessed.Inthiscase,itisvery
simpleallitdoesisinitializeavariablefcounttozero.fcounttrackshowmanylinesinthecurrentfilematchedthe
pattern.Namingtheparameterjunkshowsweknowthatbeginfile()iscalledwithaparameter,butthatwerenot
interestedinitsvalue:
functionbeginfile(junk)
{
fcount=0
}
Theendfile()functioniscalledaftereachfilehasbeenprocessed.Itaffectstheoutputonlywhentheuserwantsa
countofthenumberoflinesthatmatched.no_printistrueonlyiftheexitstatusisdesired.count_onlyistrueifline
countsaredesired.egrepthereforeonlyprintslinecountsifprintingandcountingareenabled.Theoutputformatmust
beadjusteddependinguponthenumberoffilestoprocess.Finally,fcountisaddedtototal,sothatweknowthetotal
numberoflinesthatmatchedthepattern:
functionendfile(file)
{
if(!no_print&&count_only){
if(do_filenames)
printfile":"fcount
else
printfcount
}
total+=fcount
}
TheBEGINFILEandENDFILEspecialpatterns(seeBEGINFILE/ENDFILE)couldbeused,butthentheprogramwouldbe
gawkspecific.Additionally,thisexamplewaswrittenbeforegawkacquiredBEGINFILEandENDFILE.
Thefollowingruledoesmostoftheworkofmatchinglines.Thevariablematchesistrueifthelinematchedthepattern.
Iftheuserwantslinesthatdidnotmatch,thesenseofmatchesisinvertedusingthe!operator.fcountisincremented
withthevalueofmatches,whichiseitheroneorzero,dependinguponasuccessfulorunsuccessfulmatch.Iftheline
doesnotmatch,thenextstatementjustmovesontothenextrecord.
Anumberofadditionaltestsaremade,buttheyareonlydoneifwearenotcountinglines.First,iftheuseronlywants
theexitstatus(no_printistrue),thenitisenoughtoknowthatonelineinthisfilematched,andwecanskipontothe
nextfilewithnextfile.Similarly,ifweareonlyprintingfilenames,wecanprintthefilename,andthenskiptothe
nextfilewithnextfile.Finally,eachlineisprinted,withaleadingfilenameandcolonifnecessary:
226/479
29/7/2016
TheGNUAwkUsersGuide
{
matches=($0~pattern)
if(invert)
matches=!matches
fcount+=matches#1or0
if(!matches)
next
if(!count_only){
if(no_print)
nextfile
if(filenames_only){
printFILENAME
nextfile
}
if(do_filenames)
printFILENAME":"$0
else
print
}
}
TheENDruletakescareofproducingthecorrectexitstatus.Iftherearenomatches,theexitstatusisoneotherwise,itis
zero:
END{
exit(total==0)
}
Theusage()functionprintsausagemessageincaseofinvalidoptions,andthenexits:
functionusage()
{
print("Usage:egrep[csvil][epat][files...]")>"/dev/stderr"
print("\n\tegrep[csvil]pat[files...]")>"/dev/stderr"
exit1
}
Next:SplitProgram,Previous:EgrepProgram,Up:Clones[Contents][Index]
11.2.3PrintingOutUserInformation
TheidutilitylistsausersrealandeffectiveuserIDnumbers,realandeffectivegroupIDnumbers,andtheusersgroup
set,ifany.idonlyprintstheeffectiveuserIDandgroupIDiftheyaredifferentfromtherealones.Ifpossible,idalso
suppliesthecorrespondinguserandgroupnames.Theoutputmightlooklikethis:
$id
|uid=1000(arnold)gid=1000(arnold)groups=1000(arnold),4(adm),7(lp),27(sudo)
ThisinformationispartofwhatisprovidedbygawksPROCINFOarray(seeBuiltinVariables).However,theidutility
providesamorepalatableoutputthanjustindividualnumbers.
Hereisasimpleversionofidwritteninawk.Itusestheuserdatabaselibraryfunctions(seePasswdFunctions)andthe
groupdatabaselibraryfunctions(seeGroupFunctions)fromLibraryFunctions.
Theprogramisfairlystraightforward.AlltheworkisdoneintheBEGINrule.TheuserandgroupIDnumbersare
obtainedfromPROCINFO.Thecodeisrepetitive.TheentryintheuserdatabasefortherealuserIDnumberissplitinto
partsatthe:.Thenameisthefirstfield.SimilarcodeisusedfortheeffectiveuserIDnumberandthegroupnumbers:
#id.awkimplementidinawk
227/479
29/7/2016
TheGNUAwkUsersGuide
#
#Requiresuserandgrouplibraryfunctions
#outputis:
#uid=12(foo)euid=34(bar)gid=3(baz)\
#egid=5(blat)groups=9(nine),2(two),1(one)
BEGIN{
uid=PROCINFO["uid"]
euid=PROCINFO["euid"]
gid=PROCINFO["gid"]
egid=PROCINFO["egid"]
printf("uid=%d",uid)
pw=getpwuid(uid)
pr_first_field(pw)
if(euid!=uid){
printf("euid=%d",euid)
pw=getpwuid(euid)
pr_first_field(pw)
}
printf("gid=%d",gid)
pw=getgrgid(gid)
pr_first_field(pw)
if(egid!=gid){
printf("egid=%d",egid)
pw=getgrgid(egid)
pr_first_field(pw)
}
for(i=1;("group"i)inPROCINFO;i++){
if(i==1)
printf("groups=")
group=PROCINFO["group"i]
printf("%d",group)
pw=getgrgid(group)
pr_first_field(pw)
if(("group"(i+1))inPROCINFO)
printf(",")
}
print""
}
functionpr_first_field(str,a)
{
if(str!=""){
split(str,a,":")
printf("(%s)",a[1])
}
}
Thetestintheforloopisworthnoting.AnysupplementarygroupsinthePROCINFOarrayhavetheindices"group1"
through"groupN"forsomeN(i.e.,thetotalnumberofsupplementarygroups).However,wedontknowinadvancehow
manyofthesegroupsthereare.
Thisloopworksbystartingatone,concatenatingthevaluewith"group",andthenusingintoseeifthatvalueisinthe
array(seeReferencetoElements).Eventually,iisincrementedpastthelastgroupinthearrayandtheloopexits.
Theloopisalsocorrectiftherearenosupplementarygroupsthentheconditionisfalsethefirsttimeitstested,andthe
loopbodyneverexecutes.
Thepr_first_field()functionsimplyisolatesoutsomecodethatisusedrepeatedly,makingthewholeprogramshorter
andcleaner.Inparticular,movingthecheckfortheemptystringintothisfunctionsavesseverallinesofcode.
228/479
29/7/2016
TheGNUAwkUsersGuide
Next:TeeProgram,Previous:IdProgram,Up:Clones[Contents][Index]
11.2.4SplittingaLargeFileintoPieces
Thesplitprogramsplitslargetextfilesintosmallerpieces.Usageisasfollows:73
split[count][file][prefix]
Bydefault,theoutputfilesarenamedxaa,xab,andsoon.Eachfilehas1,000linesinit,withthelikelyexceptionofthe
lastfile.Tochangethenumberoflinesineachfile,supplyanumberonthecommandlineprecededwithaminussign
(e.g.,500forfileswith500linesintheminsteadof1,000).Tochangethenamesoftheoutputfilestosomethinglike
myfileaa,myfileab,andsoon,supplyanadditionalargumentthatspecifiesthefilenameprefix.
Hereisaversionofsplitinawk.Itusestheord()andchr()functionspresentedinOrdinalFunctions.
Theprogramfirstsetsitsdefaults,andthenteststomakesuretherearenottoomanyarguments.Itthenlooksateach
argumentinturn.Thefirstargumentcouldbeaminussignfollowedbyanumber.Ifitis,thishappenstolooklikea
negativenumber,soitismadepositive,andthatisthecountoflines.Thedatafilenameisskippedoverandthefinal
argumentisusedastheprefixfortheoutputfilenames:
#split.awkdosplitinawk
#
#Requiresord()andchr()libraryfunctions
#usage:split[count][file][outname]
BEGIN{
outfile="x"#default
count=1000
if(ARGC>4)
usage()
i=1
if(iinARGV&&ARGV[i]~/^[[:digit:]]+$/){
count=ARGV[i]
ARGV[i]=""
i++
}
#testargvincasereadingfromstdininsteadoffile
if(iinARGV)
i++#skipdatafilename
if(iinARGV){
outfile=ARGV[i]
ARGV[i]=""
}
s1=s2="a"
out=(outfiles1s2)
}
Thenextruledoesmostofthework.tcount(temporarycount)trackshowmanylineshavebeenprintedtotheoutput
filesofar.Ifitisgreaterthancount,itistimetoclosethecurrentfileandstartanewone.s1ands2trackthecurrent
suffixesforthefilename.Iftheyarebothz,thefileisjusttoobig.Otherwise,s1movestothenextletterinthe
alphabetands2startsoveragainata:
{
if(++tcount>count){
close(out)
if(s2=="z"){
if(s1=="z"){
printf("split:%sistoolargetosplit\n",
FILENAME)>"/dev/stderr"
exit1
}
229/479
29/7/2016
TheGNUAwkUsersGuide
s1=chr(ord(s1)+1)
s2="a"
}
else
s2=chr(ord(s2)+1)
out=(outfiles1s2)
tcount=1
}
print>out
}
Theusage()functionsimplyprintsanerrormessageandexits:
functionusage()
{
print("usage:split[num][file][outname]")>"/dev/stderr"
exit1
}
ThisprogramisabitsloppyitreliesonawktoautomaticallyclosethelastfileinsteadofdoingitinanENDrule.Italso
assumesthatlettersarecontiguousinthecharacterset,whichisnttrueforEBCDICsystems.
Next:UniqProgram,Previous:SplitProgram,Up:Clones[Contents][Index]
11.2.5DuplicatingOutputintoMultipleFiles
Theteeprogramisknownasapipefitting.teecopiesitsstandardinputtoitsstandardoutputandalsoduplicatesitto
thefilesnamedonthecommandline.Itsusageisasfollows:
tee[a]file
Theaoptiontellsteetoappendtothenamedfiles,insteadoftruncatingthemandstartingover.
TheBEGINrulefirstmakesacopyofallthecommandlineargumentsintoanarraynamedcopy.ARGV[0]isnotneeded,
soitisnotcopied.teecannotuseARGVdirectly,becauseawkattemptstoprocesseachfilenameinARGVasinputdata.
Ifthefirstargumentisa,thentheflagvariableappendissettotrue,andbothARGV[1]andcopy[1]aredeleted.IfARGCis
lessthantwo,thennofilenamesweresuppliedandteeprintsausagemessageandexits.Finally,awkisforcedtoread
thestandardinputbysettingARGV[1]to""andARGCtotwo:
#tee.awkteeinawk
#
#Copystandardinputtoallnamedoutputfiles.
#Appendcontentifaoptionissupplied.
#
BEGIN{
for(i=1;i<ARGC;i++)
copy[i]=ARGV[i]
if(ARGV[1]=="a"){
append=1
deleteARGV[1]
deletecopy[1]
ARGC
}
if(ARGC<2){
print"usage:tee[a]file...">"/dev/stderr"
exit1
}
ARGV[1]=""
ARGC=2
}
230/479
29/7/2016
TheGNUAwkUsersGuide
Thefollowingsingleruledoesallthework.Becausethereisnopattern,itisexecutedforeachlineofinput.Thebody
oftherulesimplyprintsthelineintoeachfileonthecommandline,andthentothestandardoutput:
{
#movingtheifoutsidetheloopmakesitrunfaster
if(append)
for(iincopy)
print>>copy[i]
else
for(iincopy)
print>copy[i]
print
}
Itisalsopossibletowritetheloopthisway:
for(iincopy)
if(append)
print>>copy[i]
else
print>copy[i]
Thisismoreconcise,butitisalsolessefficient.Theifistestedforeachrecordandforeachoutputfile.By
duplicatingtheloopbody,theifisonlytestedonceforeachinputrecord.IfthereareNinputrecordsandMoutput
files,thefirstmethodonlyexecutesNifstatements,whilethesecondexecutesN*Mifstatements.
Finally,theENDrulecleansupbyclosingalltheoutputfiles:
END{
for(iincopy)
close(copy[i])
}
Next:WcProgram,Previous:TeeProgram,Up:Clones[Contents][Index]
11.2.6PrintingNonduplicatedLinesofText
Theuniqutilityreadssortedlinesofdataonitsstandardinput,andbydefaultremovesduplicatelines.Inotherwords,it
onlyprintsuniquelineshencethename.uniqhasanumberofoptions.Theusageisasfollows:
uniq[udc[n]][+n][inputfile[outputfile]]
Theoptionsforuniqare:
d
Printonlyrepeated(duplicated)lines.
u
Printonlynonrepeated(unique)lines.
c
Countlines.Thisoptionoverridesdandu.Bothrepeatedandnonrepeatedlinesarecounted.
n
Skipnfieldsbeforecomparinglines.Thedefinitionoffieldsissimilartoawksdefault:nonwhitespacecharacters
separatedbyrunsofspacesand/orTABs.
+n
231/479
29/7/2016
TheGNUAwkUsersGuide
Skipncharactersbeforecomparinglines.Anyfieldsspecifiedwithnareskippedfirst.
inputfile
Dataisreadfromtheinputfilenamedonthecommandline,insteadoffromthestandardinput.
outputfile
Thegeneratedoutputissenttothenamedoutputfile,insteadoftothestandardoutput.
Normallyuniqbehavesasifboththedanduoptionsareprovided.
uniqusesthegetopt()libraryfunction(seeGetoptFunction)andthejoin()libraryfunction(seeJoinFunction).
Theprogrambeginswithausage()functionandthenabriefoutlineoftheoptionsandtheirmeaningsincomments.
TheBEGINruledealswiththecommandlineargumentsandoptions.Itusesatricktogetgetopt()tohandleoptionsof
theform25,treatingsuchanoptionastheoptionletter2withanargumentof5.Ifindeedtwoormoredigitsare
supplied(Optarglookslikeanumber),Optargisconcatenatedwiththeoptiondigitandthentheresultisaddedtozero
tomakeitintoanumber.Ifthereisonlyonedigitintheoption,thenOptargisnotneeded.Inthiscase,Optindmustbe
decrementedsothatgetopt()processesitnexttime.Thiscodeisadmittedlyabittricky.
Ifnooptionsaresupplied,thenthedefaultistaken,toprintbothrepeatedandnonrepeatedlines.Theoutputfile,if
provided,isassignedtooutputfile.Earlyon,outputfileisinitializedtothestandardoutput,/dev/stdout:
#uniq.awkdouniqinawk
#
#Requiresgetopt()andjoin()libraryfunctions
functionusage()
{
print("Usage:uniq[udc[n]][+n][in[out]]")>"/dev/stderr"
exit1
}
#ccountlines.overridesdandu
#donlyrepeatedlines
#uonlynonrepeatedlines
#nskipnfields
#+nskipncharacters,skipfieldsfirst
BEGIN{
count=1
outputfile="/dev/stdout"
opts="udc0:1:2:3:4:5:6:7:8:9:"
while((c=getopt(ARGC,ARGV,opts))!=1){
if(c=="u")
non_repeated_only++
elseif(c=="d")
repeated_only++
elseif(c=="c")
do_count++
elseif(index("0123456789",c)!=0){
#getopt()requiresargstooptions
#thismessesusupforthingslike5
if(Optarg~/^[[:digit:]]+$/)
fcount=(cOptarg)+0
else{
fcount=c+0
Optind
}
}else
usage()
}
if(ARGV[Optind]~/^\+[[:digit:]]+$/){
charcount=substr(ARGV[Optind],2)+0
232/479
29/7/2016
TheGNUAwkUsersGuide
Optind++
}
ARGV[i]=""
if(repeated_only==0&&non_repeated_only==0)
repeated_only=non_repeated_only=1
if(ARGCOptind==2){
outputfile=ARGV[ARGC1]
ARGV[ARGC1]=""
}
}
Thefollowingfunction,are_equal(),comparesthecurrentline,$0,tothepreviousline,last.Ithandlesskippingfields
andcharacters.Ifnofieldcountandnocharactercountarespecified,are_equal()returnsoneorzerodependingupon
theresultofasimplestringcomparisonoflastand$0.
Otherwise,thingsgetmorecomplicated.Iffieldshavetobeskipped,eachlineisbrokenintoanarrayusingsplit()
(seeStringFunctions)thedesiredfieldsarethenjoinedbackintoalineusingjoin().Thejoinedlinesarestoredin
clastandcline.Ifnofieldsareskipped,clastandclinearesettolastand$0,respectively.Finally,ifcharactersare
skipped,substr()isusedtostripofftheleadingcharcountcharactersinclastandcline.Thetwostringsarethen
comparedandare_equal()returnstheresult:
functionare_equal(n,m,clast,cline,alast,aline)
{
if(fcount==0&&charcount==0)
return(last==$0)
if(fcount>0){
n=split(last,alast)
m=split($0,aline)
clast=join(alast,fcount+1,n)
cline=join(aline,fcount+1,m)
}else{
clast=last
cline=$0
}
if(charcount){
clast=substr(clast,charcount+1)
cline=substr(cline,charcount+1)
}
return(clast==cline)
}
Thefollowingtworulesarethebodyoftheprogram.Thefirstoneisexecutedonlyfortheveryfirstlineofdata.Itsets
lastequalto$0,sothatsubsequentlinesoftexthavesomethingtobecomparedto.
Thesecondruledoesthework.Thevariableequalisoneorzero,dependingupontheresultsofare_equal()s
comparison.Ifuniqiscountingrepeatedlines,andthelinesareequal,thenitincrementsthecountvariable.Otherwise,
itprintsthelineandresetscount,becausethetwolinesarenotequal.
Ifuniqisnotcounting,andifthelinesareequal,countisincremented.Nothingisprinted,asthepointistoremove
duplicates.Otherwise,ifuniqiscountingrepeatedlinesandmorethanonelineisseen,orifuniqiscounting
nonrepeatedlinesandonlyonelineisseen,thenthelineisprinted,andcountisreset.
Finally,similarlogicisusedintheENDruletoprintthefinallineofinputdata:
NR==1{
last=$0
next
}
233/479
29/7/2016
TheGNUAwkUsersGuide
{
equal=are_equal()
if(do_count){#overridesdandu
if(equal)
count++
else{
printf("%4d%s\n",count,last)>outputfile
last=$0
count=1#reset
}
next
}
if(equal)
count++
else{
if((repeated_only&&count>1)||
(non_repeated_only&&count==1))
printlast>outputfile
last=$0
count=1
}
}
END{
if(do_count)
printf("%4d%s\n",count,last)>outputfile
elseif((repeated_only&&count>1)||
(non_repeated_only&&count==1))
printlast>outputfile
close(outputfile)
}
Previous:UniqProgram,Up:Clones[Contents][Index]
11.2.7CountingThings
Thewc(wordcount)utilitycountslines,words,andcharactersinoneormoreinputfiles.Itsusageisasfollows:
wc[lwc][files]
Ifnofilesarespecifiedonthecommandline,wcreadsitsstandardinput.Iftherearemultiplefiles,italsoprintstotal
countsforallthefiles.Theoptionsandtheirmeaningsareasfollows:
l
Countonlylines.
w
Countonlywords.Awordisacontiguoussequenceofnonwhitespacecharacters,separatedbyspacesand/or
TABs.Luckily,thisisthenormalwayawkseparatesfieldsinitsinputdata.
c
Countonlycharacters.
Implementingwcinawkisparticularlyelegant,becauseawkdoesalotoftheworkforusitsplitslinesintowords(i.e.,
fields)andcountsthem,itcountslines(i.e.,records),anditcaneasilytellushowlongalineis.
Thisprogramusesthegetopt()libraryfunction(seeGetoptFunction)andthefiletransitionfunctions(seeFiletrans
Function).
234/479
29/7/2016
TheGNUAwkUsersGuide
Thisversionhasonenotabledifferencefromtraditionalversionsofwc:italwaysprintsthecountsintheorderlines,
words,andcharacters.Traditionalversionsnotetheorderofthel,w,andcoptionsonthecommandline,andprint
thecountsinthatorder.
TheBEGINruledoestheargumentprocessing.Thevariableprint_totalistrueifmorethanonefileisnamedonthe
commandline:
#wc.awkcountlines,words,characters
#Options:
#lonlycountlines
#wonlycountwords
#conlycountcharacters
#
#Defaultistocountlines,words,characters
#
#Requiresgetopt()andfiletransitionlibraryfunctions
BEGIN{
#letgetopt()printamessageabout
#invalidoptions.weignorethem
while((c=getopt(ARGC,ARGV,"lwc"))!=1){
if(c=="l")
do_lines=1
elseif(c=="w")
do_words=1
elseif(c=="c")
do_chars=1
}
ARGV[i]=""
#ifnooptions,doall
if(!do_lines&&!do_words&&!do_chars)
do_lines=do_words=do_chars=1
print_total=(ARGCi>2)
}
Thebeginfile()functionissimpleitjustresetsthecountsoflines,words,andcharacterstozero,andsavesthecurrent
filenameinfname:
functionbeginfile(file)
{
lines=words=chars=0
fname=FILENAME
}
Theendfile()functionaddsthecurrentfilesnumberstotherunningtotalsoflines,words,andcharacters.Itthen
printsoutthosenumbersforthefilethatwasjustread.Itreliesonbeginfile()toresetthenumbersforthefollowing
datafile:
functionendfile(file)
{
tlines+=lines
twords+=words
tchars+=chars
if(do_lines)
printf"\t%d",lines
if(do_words)
printf"\t%d",words
if(do_chars)
printf"\t%d",chars
printf"\t%s\n",fname
}
235/479
29/7/2016
TheGNUAwkUsersGuide
Thereisonerulethatisexecutedforeachline.Itaddsthelengthoftherecord,plusone,tochars.74Addingoneplus
therecordlengthisneededbecausethenewlinecharacterseparatingrecords(thevalueofRS)isnotpartoftherecord
itself,andthusnotincludedinitslength.Next,linesisincrementedforeachlineread,andwordsisincrementedbythe
valueofNF,whichisthenumberofwordsonthisline:
#doperline
{
chars+=length($0)+1#getnewline
lines++
words+=NF
}
Finally,theENDrulesimplyprintsthetotalsforallthefiles:
END{
if(print_total){
if(do_lines)
printf"\t%d",tlines
if(do_words)
printf"\t%d",twords
if(do_chars)
printf"\t%d",tchars
print"\ttotal"
}
}
Next:ProgramsSummary,Previous:Clones,Up:SamplePrograms[Contents][Index]
11.3AGrabBagofawkPrograms
Thissectionisalargegrabbagofmiscellaneousprograms.Wehopeyoufindthembothinterestingandenjoyable.
DupwordProgram: Findingduplicatedwordsinadocument.
AlarmProgram: Analarmclock.
TranslateProgram: Aprogramsimilartothetrutility.
LabelsProgram: Printingmailinglabels.
WordSorting:
Aprogramtoproduceawordusagecount.
HistorySorting: Eliminatingduplicateentriesfromahistoryfile.
ExtractProgram: PullingoutprogramsfromTexinfosourcefiles.
SimpleSed:
ASimpleStreamEditor.
IgawkProgram: Awrapperforawkthatincludesfiles.
AnagramProgram: Findinganagramsfromadictionary.
SignatureProgram: Peopledoamazingthingswithtoomuchtimeontheirhands.
Next:AlarmProgram,Up:MiscellaneousPrograms[Contents][Index]
11.3.1FindingDuplicatedWordsinaDocument
Acommonerrorwhenwritinglargeamountsofproseistoaccidentallyduplicatewords.Typicallyyouwillseethisin
textassomethinglikethetheprogramdoesthefollowingWhenthetextisonline,oftentheduplicatedwordsoccur
attheendofonelineandthebeginningofanother,makingthemverydifficulttospot.
Thisprogram,dupword.awk,scansthroughafileonelineatatimeandlooksforadjacentoccurrencesofthesameword.
Italsosavesthelastwordonaline(inthevariableprev)forcomparisonwiththefirstwordonthenextline.
Thefirsttwostatementsmakesurethatthelineisalllowercase,sothat,forexample,Theandthecompareequalto
eachother.Thenextstatementreplacesnonalphanumericandnonwhitespacecharacterswithspaces,sothatpunctuation
doesnotaffectthecomparisoneither.Thecharactersarereplacedwithspacessothatformattingcontrolsdontcreate
236/479
29/7/2016
TheGNUAwkUsersGuide
nonsensewords(e.g.,theTexinfo@code{NF}becomes codeNFifpunctuationissimplydeleted).Therecordisthen
resplitintofields,yieldingjusttheactualwordsontheline,andensuringthattherearenoemptyfields.
Iftherearenofieldsleftafterremovingallthepunctuation,thecurrentrecordisskipped.Otherwise,theprogramloops
througheachword,comparingittothepreviousone:
#dupword.awkfindduplicatewordsintext
{
$0=tolower($0)
gsub(/[^[:alnum:][:blank:]]/,"");
$0=$0#resplit
if(NF==0)
next
if($1==prev)
printf("%s:%d:duplicate%s\n",
FILENAME,FNR,$1)
for(i=2;i<=NF;i++)
if($i==$(i1))
printf("%s:%d:duplicate%s\n",
FILENAME,FNR,$i)
prev=$NF
}
Next:TranslateProgram,Previous:DupwordProgram,Up:MiscellaneousPrograms[Contents][Index]
11.3.2AnAlarmClockProgram
Nothingcuresinsomnialikearingingalarmclock.
ArnoldRobbins
Sleepisforwebdevelopers.
ErikQuanstrom
Thefollowingprogramisasimplealarmclockprogram.Yougiveitatimeofdayandanoptionalmessage.Atthe
specifiedtime,itprintsthemessageonthestandardoutput.Inaddition,youcangiveitthenumberoftimestorepeatthe
messageaswellasadelaybetweenrepetitions.
Thisprogramusesthegetlocaltime()functionfromGetlocaltimeFunction.
AlltheworkisdoneintheBEGINrule.Thefirstpartisargumentcheckingandsettingofdefaults:thedelay,thecount,
andthemessagetoprint.IftheusersuppliedamessagewithouttheASCIIBELcharacter(knownasthealert
character,"\a"),thenitisaddedtothemessage.(Onmanysystems,printingtheASCIIBELgeneratesanaudiblealert.
Thus,whenthealarmgoesoff,thesystemcallsattentiontoitselfincasetheuserisnotlookingatthecomputer.)Just
forachange,thisprogramusesaswitchstatement(seeSwitchStatement),buttheprocessingcouldbedonewitha
seriesofifelsestatementsinstead.Hereistheprogram:
#alarm.awksetanalarm
#
#Requiresgetlocaltime()libraryfunction
#usage:alarmtime["message"[count[delay]]]
BEGIN{
#Initialargumentsanitychecking
usage1="usage:alarmtime['message'[count[delay]]]"
usage2=sprintf("\t(%s)time::=hh:mm",ARGV[1])
if(ARGC<2){
printusage1>"/dev/stderr"
exit1
}
237/479
29/7/2016
TheGNUAwkUsersGuide
switch(ARGC){
case5:
delay=ARGV[4]+0
#fallthrough
case4:
count=ARGV[3]+0
#fallthrough
case3:
message=ARGV[2]
break
default:
if(ARGV[1]!~/[[:digit:]]?[[:digit:]]:[[:digit:]]{2}/){
exit1
}
break
}
#setdefaultsforoncewereachthedesiredtime
if(delay==0)
delay=180#3minutes
if(count==0)
count=5
if(message=="")
message=sprintf("\aItisnow%s!\a",ARGV[1])
elseif(index(message,"\a")==0)
message="\a"message"\a"
Thenextsectionofcodeturnsthealarmtimeintohoursandminutes,convertsit(ifnecessary)toa24hourclock,and
thenturnsthattimeintoacountofthesecondssincemidnight.Nextitturnsthecurrenttimeintoacountofseconds
sincemidnight.Thedifferencebetweenthetwoishowlongtowaitbeforesettingoffthealarm:
#splitupalarmtime
split(ARGV[1],atime,":")
hour=atime[1]+0#forcenumeric
minute=atime[2]+0#forcenumeric
#getcurrentbrokendowntime
getlocaltime(now)
#iftimegivenis12hourhoursandit'safterthat
#hour,e.g.,àlarm5:30'at9a.m.means5:30p.m.,
#thenadd12torealhour
if(hour<12&&now["hour"]>hour)
hour+=12
#settargettimeinsecondssincemidnight
target=(hour*60*60)+(minute*60)
#getcurrenttimeinsecondssincemidnight
current=(now["hour"]*60*60)+\
(now["minute"]*60)+now["second"]
#howlongtosleepfor
naptime=targetcurrent
if(naptime<=0){
print"alarm:timeisinthepast!">"/dev/stderr"
exit1
}
Finally,theprogramusesthesystem()function(seeI/OFunctions)tocallthesleeputility.Thesleeputilitysimply
pausesforthegivennumberofseconds.Iftheexitstatusisnotzero,theprogramassumesthatsleepwasinterrupted
andexits.IfsleepexitedwithanOKstatus(zero),thentheprogramprintsthemessageinaloop,againusingsleepto
delayforhowevermanysecondsarenecessary:
238/479
29/7/2016
TheGNUAwkUsersGuide
#zzzzzz.....goawayifinterrupted
if(system(sprintf("sleep%d",naptime))!=0)
exit1
#timetonotify!
command=sprintf("sleep%d",delay)
for(i=1;i<=count;i++){
printmessage
#ifsleepcommandinterrupted,goaway
if(system(command)!=0)
break
}
exit0
}
Next:LabelsProgram,Previous:AlarmProgram,Up:MiscellaneousPrograms[Contents][Index]
11.3.3TransliteratingCharacters
Thesystemtrutilitytransliteratescharacters.Forexample,itisoftenusedtomapuppercaselettersintolowercasefor
furtherprocessing:
generatedata|tr'AZ''az'|processdata
trrequirestwolistsofcharacters.75Whenprocessingtheinput,thefirstcharacterinthefirstlistisreplacedwiththe
firstcharacterinthesecondlist,thesecondcharacterinthefirstlistisreplacedwiththesecondcharacterinthesecond
list,andsoon.Iftherearemorecharactersinthefromlistthaninthetolist,thelastcharacterofthetolistisused
fortheremainingcharactersinthefromlist.
Onceuponatime,auserproposedaddingatransliterationfunctiontogawk.Thefollowingprogramwaswrittentoprove
thatcharactertransliterationcouldbedonewithauserlevelfunction.Thisprogramisnotascompleteasthesystemtr
utility,butitdoesmostofthejob.
Thetranslateprogramwaswrittenlongbeforegawkacquiredtheabilitytospliteachcharacterinastringintoseparate
arrayelements.Thus,itmakesrepeateduseofthesubstr(),index(),andgsub()builtinfunctions(seeString
Functions).Therearetwofunctions.Thefirst,stranslate(),takesthreearguments:
from
Alistofcharactersfromwhichtotranslate
to
Alistofcharacterstowhichtotranslate
target
Thestringonwhichtodothetranslation
Associativearraysmakethetranslationpartfairlyeasy.t_arholdsthetocharacters,indexedbythefrom
characters.Thenasimpleloopgoesthroughfrom,onecharacteratatime.Foreachcharacterinfrom,ifthecharacter
appearsintarget,itisreplacedwiththecorrespondingtocharacter.
Thetranslate()functioncallsstranslate(),using$0asthetarget.Themainprogramsetstwoglobalvariables,FROM
andTO,fromthecommandline,andthenchangesARGVsothatawkreadsfromthestandardinput.
Finally,theprocessingrulesimplycallstranslate()foreachrecord:
#translate.awkdotrlikestuff
#Bugs:doesnothandlethingsliketrAZaz;ithas
239/479
29/7/2016
TheGNUAwkUsersGuide
#tobespelledout.However,if`to'isshorterthan`from',
#thelastcharacterin`to'isusedfortherestof`from'.
functionstranslate(from,to,target,lf,lt,ltarget,t_ar,i,c,
result)
{
lf=length(from)
lt=length(to)
ltarget=length(target)
for(i=1;i<=lt;i++)
t_ar[substr(from,i,1)]=substr(to,i,1)
if(lt<lf)
for(;i<=lf;i++)
t_ar[substr(from,i,1)]=substr(to,lt,1)
for(i=1;i<=ltarget;i++){
c=substr(target,i,1)
if(cint_ar)
c=t_ar[c]
result=resultc
}
returnresult
}
functiontranslate(from,to)
{
return$0=stranslate(from,to,$0)
}
#mainprogram
BEGIN{
if(ARGC<3){
print"usage:translatefromto">"/dev/stderr"
exit
}
FROM=ARGV[1]
TO=ARGV[2]
ARGC=2
ARGV[1]=""
}
{
translate(FROM,TO)
print
}
Itispossibletodocharactertransliterationinauserlevelfunction,butitisnotnecessarilyefficient,andwe(thegawk
developers)startedtoconsideraddingabuiltinfunction.However,shortlyafterwritingthisprogram,welearnedthat
BrianKernighanhadaddedthetoupper()andtolower()functionstohisawk(seeStringFunctions).Thesefunctions
handlethevastmajorityofthecaseswherecharactertransliterationisnecessary,andsowechosetosimplyaddthose
functionstogawkaswellandthenleavewellenoughalone.
Anobviousimprovementtothisprogramwouldbetosetupthet_ararrayonlyonce,inaBEGINrule.However,this
assumesthatthefromandtolistswillneverchangethroughoutthelifetimeoftheprogram.
Anotherobviousimprovementistoenabletheuseofranges,suchasaz,asallowedbythetrutility.Lookatthecode
forcut.awk(seeCutProgram)forinspiration.
Next:WordSorting,Previous:TranslateProgram,Up:MiscellaneousPrograms[Contents][Index]
11.3.4PrintingMailingLabels
Hereisarealworld76program.Thisscriptreadslistsofnamesandaddressesandgeneratesmailinglabels.Eachpage
240/479
29/7/2016
TheGNUAwkUsersGuide
oflabelshas20labelsonit,twoacrossand10down.Theaddressesareguaranteedtobenomorethanfivelinesofdata.
Eachaddressisseparatedfromthenextbyablankline.
Thebasicideaistoread20labelsworthofdata.Eachlineofeachlabelisstoredinthelinearray.Thesingleruletakes
careoffillingthelinearrayandprintingthepagewhen20labelshavebeenread.
TheBEGINrulesimplysetsRStotheemptystring,sothatawksplitsrecordsatblanklines(seeRecords).ItsetsMAXLINES
to100,because100isthemaximumnumberoflinesonthepage(20*5=100).
Mostoftheworkisdoneintheprintpage()function.Thelabellinesarestoredsequentiallyinthelinearray.Butthey
havetoprinthorizontally:line[1]nexttoline[6],line[2]nexttoline[7],andsoon.Twoloopsaccomplishthis.The
outerloop,controlledbyi,stepsthroughevery10linesofdatathisiseachrowoflabels.Theinnerloop,controlledby
j,goesthroughthelineswithintherow.Asjgoesfrom0to4,i+jisthejthlineintherow,andi+j+5istheentry
nexttoit.Theoutputendsuplookingsomethinglikethis:
line1line6
line2line7
line3line8
line4line9
line5line10
Theprintfformatstring%41sleftalignsthedataandprintsitwithinafixedwidthfield.
Asafinalnote,anextrablanklineisprintedatlines21and61,tokeeptheoutputlineduponthelabels.Thisis
dependentontheparticularbrandoflabelsinusewhentheprogramwaswritten.Youwillalsonotethattherearetwo
blanklinesatthetopandtwoblanklinesatthebottom.
TheENDrulearrangestoflushthefinalpageoflabelstheremaynothavebeenanevenmultipleof20labelsinthedata:
#labels.awkprintmailinglabels
#Eachlabelis5linesofdatathatmayhaveblanklines.
#Thelabelsheetshave2blanklinesatthetopand2at
#thebottom.
BEGIN{RS="";MAXLINES=100}
functionprintpage(i,j)
{
if(Nlines<=0)
return
printf"\n\n"#header
for(i=1;i<=Nlines;i+=10){
if(i==21||i==61)
print""
for(j=0;j<5;j++){
if(i+j>MAXLINES)
break
printf"%41s%s\n",line[i+j],line[i+j+5]
}
print""
}
printf"\n\n"#footer
deleteline
}
#mainrule
{
if(Count>=20){
printpage()
241/479
29/7/2016
TheGNUAwkUsersGuide
Count=0
Nlines=0
}
n=split($0,a,"\n")
for(i=1;i<=n;i++)
line[++Nlines]=a[i]
for(;i<=5;i++)
line[++Nlines]=""
Count++
}
END{
printpage()
}
Next:HistorySorting,Previous:LabelsProgram,Up:MiscellaneousPrograms[Contents][Index]
11.3.5GeneratingWordUsageCounts
Whenworkingwithlargeamountsoftext,itcanbeinterestingtoknowhowoftendifferentwordsappear.Forexample,
anauthormayoverusecertainwords,inwhichcaseheorshemightwishtofindsynonymstosubstituteforwordsthat
appeartoooften.Thissubsectiondevelopsaprogramforcountingwordsandpresentingthefrequencyinformationina
usefulformat.
Atfirstglance,aprogramlikethiswouldseemtodothejob:
#wordfreqfirsttry.awkprintlistofwordfrequencies
{
for(i=1;i<=NF;i++)
freq[$i]++
}
END{
for(wordinfreq)
printf"%s\t%d\n",word,freq[word]
}
Theprogramreliesonawksdefaultfieldsplittingmechanismtobreakeachlineupintowordsandusesanassociative
arraynamedfreq,indexedbyeachword,tocountthenumberoftimesthewordoccurs.IntheENDrule,itprintsthe
counts.
Thisprogramhasseveralproblemsthatpreventitfrombeingusefulonrealtextfiles:
Theawklanguageconsidersupperandlowercasecharacterstobedistinct.Therefore,bartenderand
Bartenderarenottreatedasthesameword.Thisisundesirable,becausewordsarecapitalizediftheybegin
sentencesinnormaltext,andafrequencyanalyzershouldnotbesensitivetocapitalization.
Wordsaredetectedusingtheawkconventionthatfieldsareseparatedjustbywhitespace.Othercharactersinthe
input(exceptnewlines)donthaveanyspecialmeaningtoawk.Thismeansthatpunctuationcharacterscountas
partofwords.
Theoutputdoesnotcomeoutinanyusefulorder.Youremorelikelytobeinterestedinwhichwordsoccurmost
frequentlyorinhavinganalphabetizedtableofhowfrequentlyeachwordoccurs.
Thefirstproblemcanbesolvedbyusingtolower()toremovecasedistinctions.Thesecondproblemcanbesolvedby
usinggsub()toremovepunctuationcharacters.Finally,wesolvethethirdproblembyusingthesystemsortutilityto
processtheoutputoftheawkscript.Hereisthenewversionoftheprogram:
#wordfreq.awkprintlistofwordfrequencies
{
$0=tolower($0)#removecasedistinctions
#removepunctuation
242/479
29/7/2016
TheGNUAwkUsersGuide
gsub(/[^[:alnum:]_[:blank:]]/,"",$0)
for(i=1;i<=NF;i++)
freq[$i]++
}
END{
for(wordinfreq)
printf"%s\t%d\n",word,freq[word]
}
Theregexp/[^[:alnum:]_[:blank:]]/mighthavebeenwritten/[[:punct:]]/,butthenunderscoreswouldalsobe
removed,andwewanttokeepthem.
Assumingwehavesavedthisprograminafilenamedwordfreq.awk,andthatthedataisinfile1,thefollowing
pipeline:
awkfwordfreq.awkfile1|sortk2nr
producesatableofthewordsappearinginfile1inorderofdecreasingfrequency.
Theawkprogramsuitablymassagesthedataandproducesawordfrequencytable,whichisnotordered.Theawkscripts
outputisthensortedbythesortutilityandprintedonthescreen.
Theoptionsgiventosortspecifyasortthatusesthesecondfieldofeachinputline(skippingonefield),thatthesort
keysshouldbetreatedasnumericquantities(otherwise15wouldcomebefore5),andthatthesortingshouldbedone
indescending(reverse)order.
Thesortcouldevenbedonefromwithintheprogram,bychangingtheENDactionto:
END{
sort="sortk2nr"
for(wordinfreq)
printf"%s\t%d\n",word,freq[word]|sort
close(sort)
}
Thiswayofsortingmustbeusedonsystemsthatdonothavetruepipesatthecommandline(orbatchfile)level.See
thegeneraloperatingsystemdocumentationformoreinformationonhowtousethesortprogram.
Next:ExtractProgram,Previous:WordSorting,Up:MiscellaneousPrograms[Contents][Index]
11.3.6RemovingDuplicatesfromUnsortedText
Theuniqprogram(seeUniqProgram)removesduplicatelinesfromsorteddata.
Suppose,however,youneedtoremoveduplicatelinesfromadatafilebutthatyouwanttopreservetheorderthelines
arein.Agoodexampleofthismightbeashellhistoryfile.Thehistoryfilekeepsacopyofallthecommandsyouhave
entered,anditisnotunusualtorepeatacommandseveraltimesinarow.Occasionallyyoumightwanttocompactthe
historybyremovingduplicateentries.Yetitisdesirabletomaintaintheorderoftheoriginalcommands.
Thissimpleprogramdoesthejob.Itusestwoarrays.Thedataarrayisindexedbythetextofeachline.Foreachline,
data[$0]isincremented.Ifaparticularlinehasnotbeenseenbefore,thendata[$0]iszero.Inthiscase,thetextofthe
lineisstoredinlines[count].Eachelementoflinesisauniquecommand,andtheindicesoflinesindicatetheorder
inwhichthoselinesareencountered.TheENDrulesimplyprintsoutthelines,inorder:
#histsort.awkcompactashellhistoryfile
#ThankstoByronRakitzisforthegeneralidea
{
if(data[$0]++==0)
243/479
29/7/2016
TheGNUAwkUsersGuide
lines[++count]=$0
}
END{
for(i=1;i<=count;i++)
printlines[i]
}
Thisprogramalsoprovidesafoundationforgeneratingotherusefulinformation.Forexample,usingthefollowing
printstatementintheENDruleindicateshowoftenaparticularcommandisused:
printdata[lines[i]],lines[i]
Thisworksbecausedata[$0]isincrementedeachtimealineisseen.
Next:SimpleSed,Previous:HistorySorting,Up:MiscellaneousPrograms[Contents][Index]
11.3.7ExtractingProgramsfromTexinfoSourceFiles
Boththischapterandthepreviouschapter(LibraryFunctions)presentalargenumberofawkprograms.Ifyouwantto
experimentwiththeseprograms,itistedioustotypetheminbyhand.Herewepresentaprogramthatcanextractparts
ofaTexinfoinputfileintoseparatefiles.
ThisWebpageiswritteninTexinfo,theGNUProjectsdocumentformattinglanguage.AsingleTexinfosourcefilecan
beusedtoproducebothprinteddocumentation,withTeX,andonlinedocumentation.(Texinfoisfullydocumentedin
thebookTexinfoTheGNUDocumentationFormat,availablefromtheFreeSoftwareFoundation,andalsoavailable
online.)
Forourpurposes,itisenoughtoknowthreethingsaboutTexinfoinputfiles:
Theatsymbol(@)isspecialinTexinfo,muchasthebackslash(\)isinCorawk.Literal@symbolsare
representedinTexinfosourcefilesas@@.
Commentsstartwitheither@cor@comment.Thefileextractionprogramworksbyusingspecialcommentsthat
startatthebeginningofaline.
Linescontaining@groupand@endgroupcommandsbracketexampletextthatshouldnotbesplitacrossapage
boundary.(Unfortunately,TeXisntalwayssmartenoughtodothingsexactlyright,sowehavetogiveitsome
help.)
Thefollowingprogram,extract.awk,readsthroughaTexinfosourcefileanddoestwothings,basedonthespecial
comments.Uponseeing@csystem,itrunsacommand,byextractingthecommandtextfromthecontrollineand
passingitontothesystem()function(seeI/OFunctions).Uponseeing@cfilefilename,eachsubsequentlineissent
tothefilefilename,until@cendfileisencountered.Therulesinextract.awkmatcheither@cor@commentby
lettingtheommentpartbeoptional.Linescontaining@groupand@endgrouparesimplyremoved.extract.awkuses
thejoin()libraryfunction(seeJoinFunction).
TheexampleprogramsintheonlineTexinfosourceforGAWK:EffectiveAWKProgramming(gawktexi.in)haveall
beenbracketedinsidefileandendfilelines.Thegawkdistributionusesacopyofextract.awktoextractthesample
programsandinstallmanyoftheminastandarddirectorywheregawkcanfindthem.TheTexinfofilelookssomething
likethis:
Thisprogramhasa@code{BEGIN}rule
thatprintsanicemessage:
@example
@cfileexamples/messages.awk
BEGIN@{print"Don'tpanic!"@}
@cendfile
@endexample
244/479
29/7/2016
TheGNUAwkUsersGuide
Italsoprintssomefinaladvice:
@example
@cfileexamples/messages.awk
END@{print"Alwaysavoidboredarchaeologists!"@}
@cendfile
@endexample
extract.awkbeginsbysettingIGNORECASEtoone,sothatmixedupperandlowercaselettersinthedirectiveswont
matter.
Thefirstrulehandlescallingsystem(),checkingthatacommandisgiven(NFisatleastthree)andalsocheckingthatthe
commandexitswithazeroexitstatus,signifyingOK:
#extract.awkextractfilesandrunprogramsfromTexinfofiles
BEGIN{IGNORECASE=1}
/^@c(omment)?[\t]+system/{
if(NF<3){
e=("extract:"FILENAME":"FNR)
e=(e":badlyformed`system'line")
next
}
$1=""
$2=""
stat=system($0)
if(stat!=0){
e=("extract:"FILENAME":"FNR)
e=(e":warning:systemreturned"stat)
}
}
Thevariableeisusedsothattherulefitsnicelyonthescreen.
Thesecondrulehandlesmovingdataintofiles.Itverifiesthatafilenameisgiveninthedirective.Ifthefilenamedis
notthecurrentfile,thenthecurrentfileisclosed.Keepingthecurrentfileopenuntilanewfileisencounteredallows
theuseofthe>redirectionforprintingthecontents,keepingopenfilemanagementsimple.
Theforloopdoesthework.Itreadslinesusinggetline(seeGetline).Foranunexpectedendoffile,itcallsthe
unexpected_eof()function.Ifthelineisanendfileline,thenitbreaksoutoftheloop.Ifthelineisan@groupor
@endgroupline,thenitignoresitandgoesontothenextline.Similarly,commentswithinexamplesarealsoignored.
Mostoftheworkisinthefollowingfewlines.Ifthelinehasno@symbols,theprogramcanprintitdirectly.
Otherwise,eachleading@mustbestrippedoff.Toremovethe@symbols,thelineissplitintoseparateelementsof
thearraya,usingthesplit()function(seeStringFunctions).The@symbolisusedastheseparatorcharacter.Each
elementofathatisemptyindicatestwosuccessive@symbolsintheoriginalline.Foreachtwoemptyelements(@@
intheoriginalfile),wehavetoaddasingle@symbolbackin.
Whentheprocessingofthearrayisfinished,join()iscalledwiththevalueofSUBSEP(seeMultidimensional),torejoin
thepiecesbackintoasingleline.Thatlineisthenprintedtotheoutputfile:
/^@c(omment)?[\t]+file/{
if(NF!=3){
e=("extract:"FILENAME":"FNR":badlyformed`file'line")
next
}
if($3!=curfile){
if(curfile!="")
close(curfile)
curfile=$3
245/479
29/7/2016
TheGNUAwkUsersGuide
}
for(;;){
if((getlineline)<=0)
unexpected_eof()
if(line~/^@c(omment)?[\t]+endfile/)
break
elseif(line~/^@(end[\t]+)?group/)
continue
elseif(line~/^@c(omment+)?[\t]+/)
continue
if(index(line,"@")==0){
printline>curfile
continue
}
n=split(line,a,"@")
#ifa[1]=="",meansleading@,
#don'taddonebackin.
for(i=2;i<=n;i++){
if(a[i]==""){#wasan@@
a[i]="@"
if(a[i+1]=="")
i++
}
}
printjoin(a,1,n,SUBSEP)>curfile
}
}
Animportantthingtonoteistheuseofthe>redirection.Outputdonewith>onlyopensthefileonceitstaysopen
andsubsequentoutputisappendedtothefile(seeRedirection).Thismakesiteasytomixprogramtextandexplanatory
proseforthesamesamplesourcefile(ashasbeendonehere!)withoutanyhassle.Thefileisonlyclosedwhenanew
datafilenameisencounteredorattheendoftheinputfile.
Finally,thefunctionunexpected_eof()printsanappropriateerrormessageandthenexits.TheENDrulehandlesthefinal
cleanup,closingtheopenfile:
functionunexpected_eof()
{
printf("extract:%s:%d:unexpectedEOForerror\n",
FILENAME,FNR)>"/dev/stderr"
exit1
}
END{
if(curfile)
close(curfile)
}
Next:IgawkProgram,Previous:ExtractProgram,Up:MiscellaneousPrograms[Contents][Index]
11.3.8ASimpleStreamEditor
Thesedutilityisastreameditor,aprogramthatreadsastreamofdata,makeschangestoit,andpassesiton.Itisoften
usedtomakeglobalchangestoalargefileortoastreamofdatageneratedbyapipelineofcommands.Althoughsedis
acomplicatedprograminitsownright,itsmostcommonuseistoperformglobalsubstitutionsinthemiddleofa
pipeline:
command1<orig.data|sed's/old/new/g'|command2>result
Here,s/old/new/gtellssedtolookfortheregexpoldoneachinputlineandgloballyreplaceitwiththetextnew
(i.e.,alltheoccurrencesonaline).Thisissimilartoawksgsub()function(seeStringFunctions).
Thefollowingprogram,awksed.awk,acceptsatleasttwocommandlinearguments:thepatterntolookforandthetextto
246/479
29/7/2016
TheGNUAwkUsersGuide
replaceitwith.Anyadditionalargumentsaretreatedasdatafilenamestoprocess.Ifnoneareprovided,thestandard
inputisused:
#awksed.awkdos/foo/bar/gusingjustprint
#ThankstoMichaelBrennanfortheidea
functionusage()
{
print"usage:awksedpatrepl[files...]">"/dev/stderr"
exit1
}
BEGIN{
#validatearguments
if(ARGC<3)
usage()
RS=ARGV[1]
ORS=ARGV[2]
#don'tuseargumentsasfiles
ARGV[1]=ARGV[2]=""
}
#lookma,nohands!
{
if(RT=="")
printf"%s",$0
else
print
}
TheprogramreliesongawksabilitytohaveRSbearegexp,aswellasonthesettingofRTtotheactualtextthat
terminatestherecord(seeRecords).
TheideaistohaveRSbethepatterntolookfor.gawkautomaticallysets$0tothetextbetweenmatchesofthepattern.
Thisistextthatwewanttokeep,unmodified.Then,bysettingORStothereplacementtext,asimpleprintstatement
outputsthetextwewanttokeep,followedbythereplacementtext.
Thereisonewrinkletothisscheme,whichiswhattodoifthelastrecorddoesntendwithtextthatmatchesRS.Usinga
printstatementunconditionallyprintsthereplacementtext,whichisnotcorrect.However,ifthefiledidnotendintext
thatmatchesRS,RTissettothenullstring.Inthiscase,wecanprint$0usingprintf(seePrintf).
TheBEGINrulehandlesthesetup,checkingfortherightnumberofargumentsandcallingusage()ifthereisaproblem.
ThenitsetsRSandORSfromthecommandlineargumentsandsetsARGV[1]andARGV[2]tothenullstring,sothatthey
arenottreatedasfilenames(seeARGCandARGV).
Theusage()functionprintsanerrormessageandexits.Finally,thesinglerulehandlestheprintingschemeoutlined
earlier,usingprintorprintfasappropriate,dependinguponthevalueofRT.
Next:AnagramProgram,Previous:SimpleSed,Up:MiscellaneousPrograms[Contents][Index]
11.3.9AnEasyWaytoUseLibraryFunctions
InIncludeFiles,wesawhowgawkprovidesabuiltinfileinclusioncapability.However,thisisagawkextension.This
sectionprovidesthemotivationformakingfileinclusionavailableforstandardawk,andshowshowtodoitusinga
combinationofshellandawkprogramming.
Usinglibraryfunctionsinawkcanbeverybeneficial.Itencouragescodereuseandthewritingofgeneralfunctions.
Programsaresmallerandthereforeclearer.However,usinglibraryfunctionsisonlyeasywhenwritingawkprogramsit
ispainfulwhenrunningthem,requiringmultiplefoptions.Ifgawkisunavailable,thensotooistheAWKPATH
247/479
29/7/2016
TheGNUAwkUsersGuide
environmentvariableandtheabilitytoputawkfunctionsintoalibrarydirectory(seeOptions).Itwouldbenicetobe
abletowriteprogramsinthefollowingmanner:
#libraryfunctions
@includegetopt.awk
@includejoin.awk
#mainprogram
BEGIN{
while((c=getopt(ARGC,ARGV,"a:b:cde"))!=1)
Thefollowingprogram,igawk.sh,providesthisservice.ItsimulatesgawkssearchingoftheAWKPATHvariableandalso
allowsnestedincludes(i.e.,afilethatisincludedwith@includecancontainfurther@includestatements).igawkmakes
anefforttoonlyincludefilesonce,sothatnestedincludesdontaccidentallyincludealibraryfunctiontwice.
igawkshouldbehavejustlikegawkexternally.Thismeansitshouldacceptallofgawkscommandlinearguments,
includingtheabilitytohavemultiplesourcefilesspecifiedviafandtheabilitytomixcommandlineandlibrary
sourcefiles.
TheprogramiswrittenusingthePOSIXShell(sh)commandlanguage.77Itworksasfollows:
1.Loopthroughthearguments,savinganythingthatdoesntrepresentawksourcecodeforlater,whentheexpanded
programisrun.
2.Foranyargumentsthatdorepresentawktext,puttheargumentsintoashellvariablethatwillbeexpanded.There
aretwocases:
1.Literaltext,providedwitheorsource.Thistextisjustappendeddirectly.
2.Sourcefilenames,providedwithf.Weuseaneattrickandappend@includefilenametotheshell
variablescontents.Becausethefileinclusionprogramworksthewaygawkdoes,thisgetsthetextofthe
fileincludedintheprogramatthecorrectpoint.
3.Runanawkprogram(naturally)overtheshellvariablescontentstoexpand@includestatements.Theexpanded
programisplacedinasecondshellvariable.
4.Runtheexpandedprogramwithgawkandanyotheroriginalcommandlineargumentsthattheusersupplied(such
asthedatafilenames).
Thisprogramusesshellvariablesextensively:forstoringcommandlineargumentsandthetextoftheawkprogramthat
willexpandtheusersprogram,fortheusersoriginalprogram,andfortheexpandedprogram.Doingsoremovessome
potentialproblemsthatmightarisewerewetousetemporaryfilesinstead,atthecostofmakingthescriptsomewhat
morecomplicated.
Theinitialpartoftheprogramturnsonshelltracingifthefirstargumentisdebug.
Thenextpartloopsthroughallthecommandlinearguments.Thereareseveralcasesofinterest:
Thisendstheargumentstoigawk.Anythingelseshouldbepassedontotheusersawkprogramwithoutbeing
evaluated.
W
Thisindicatesthatthenextoptionisspecifictogawk.Tomakeargumentprocessingeasier,theWisappendedto
thefrontoftheremainingargumentsandtheloopcontinues.(Thisisanshprogrammingtrick.Dontworryabout
itifyouarenotfamiliarwithsh.)
v,F
Thesearesavedandpassedontogawk.
248/479
29/7/2016
TheGNUAwkUsersGuide
f,file,file=,Wfile=
Thefilenameisappendedtotheshellvariableprogramwithan@includestatement.Theexprutilityisusedto
removetheleadingoptionpartoftheargument(e.g.,file=).(Typicalshusagewouldbetousetheechoand
sedutilitiestodothiswork.Unfortunately,someversionsofechoevaluateescapesequencesintheirarguments,
possiblymanglingtheprogramtext.Usingexpravoidsthisproblem.)
source,source=,Wsource=
Thesourcetextisappendedtoprogram.
version,Wversion
igawkprintsitsversionnumber,runsgawkversiontogetthe gawkversioninformation,andthenexits.
Ifnoneofthef,file,Wfile,source,orWsourceargumentsaresupplied,thenthefirstnonoptionargument
shouldbetheawkprogram.Iftherearenocommandlineargumentsleft,igawkprintsanerrormessageandexits.
Otherwise,thefirstargumentisappendedtoprogram.Inanycase,aftertheargumentshavebeenprocessed,theshell
variableprogramcontainsthecompletetextoftheoriginalawkprogram.
Theprogramisasfollows:
#!/bin/sh
#igawklikegawkbutdo@includeprocessing
if["$1"=debug]
then
setx
shift
fi
#Aliteralnewline,sothatprogramtextisformattedcorrectly
n='
'
#Initializevariablestoempty
program=
opts=
while[$#ne0]#loopoverarguments
do
case$1in
)shift
break;;
W)shift
#The${x?'messagehere'}constructprintsa
#diagnosticif$xisthenullstring
setW"${@?'missingoperand'}"
continue;;
[vF])opts="$opts$1'${2?'missingoperand'}'"
shift;;
[vF]*)opts="$opts'$1'";;
f)program="$program$n@include${2?'missingoperand'}"
shift;;
f*)f=$(expr"$1":'f$.*$')
program="$program$n@include$f";;
[W]file=*)
f=$(expr"$1":'.file=$.*$')
program="$program$n@include$f";;
249/479
29/7/2016
TheGNUAwkUsersGuide
[W]file)
program="$program$n@include${2?'missingoperand'}"
shift;;
[W]source=*)
t=$(expr"$1":'.source=$.*$')
program="$program$n$t";;
[W]source)
program="$program$n${2?'missingoperand'}"
shift;;
[W]version)
echoigawk:version3.01>&2
gawkversion
exit0;;
[W]*)opts="$opts'$1'";;
*)break;;
esac
shift
done
if[z"$program"]
then
program=${1?'missingprogram'}
shift
fi
#Atthispoint,`program'hastheprogram.
Theawkprogramtoprocess@includedirectivesisstoredintheshellvariableexpand_prog.Doingthiskeepstheshell
scriptreadable.Theawkprogramreadsthroughtheusersprogram,onelineatatime,usinggetline(seeGetline).The
inputfilenamesand@includestatementsaremanagedusingastack.Aseach@includeisencountered,thecurrentfile
nameispushedontothestackandthefilenamedinthe@includedirectivebecomesthecurrentfilename.Aseachfile
isfinished,thestackispopped,andthepreviousinputfilebecomesthecurrentinputfileagain.Theprocessisstarted
bymakingtheoriginalfilethefirstoneonthestack.
Thepathto()functiondoestheworkoffindingthefullpathtoafile.Itsimulatesgawksbehaviorwhensearchingthe
AWKPATHenvironmentvariable(seeAWKPATHVariable).Ifafilenamehasa/init,nopathsearchisdone.Similarly,
ifthefilenameis"",thenthatstringisusedasis.Otherwise,thefilenameisconcatenatedwiththenameofeach
directoryinthepath,andanattemptismadetoopenthegeneratedfilename.Theonlywaytotestifafilecanbereadin
awkistogoaheadandtrytoreaditwithgetlinethisiswhatpathto()does.78Ifthefilecanberead,itisclosedandthe
filenameisreturned:
expand_prog='
functionpathto(file,i,t,junk)
{
if(index(file,"/")!=0)
returnfile
if(file=="")
returnfile
for(i=1;i<=ndirs;i++){
t=(pathlist[i]"/"file)
if((getlinejunk<t)>0){
#foundit
close(t)
returnt
}
}
250/479
29/7/2016
TheGNUAwkUsersGuide
return""
}
ThemainprogramiscontainedinsideoneBEGINrule.Thefirstthingitdoesissetupthepathlistarraythatpathto()
uses.Aftersplittingthepathon:,nullelementsarereplacedwith".",whichrepresentsthecurrentdirectory:
BEGIN{
path=ENVIRON["AWKPATH"]
ndirs=split(path,pathlist,":")
for(i=1;i<=ndirs;i++){
if(pathlist[i]=="")
pathlist[i]="."
}
ThestackisinitializedwithARGV[1],whichwillbe"/dev/stdin".Themainloopcomesnext.Inputlinesarereadin
succession.Linesthatdonotstartwith@includeareprintedverbatim.Ifthelinedoesstartwith@include,thefilename
isin$2.pathto()iscalledtogeneratethefullpath.Ifitcannot,thentheprogramprintsanerrormessageandcontinues.
Thenextthingtocheckisifthefileisincludedalready.Theprocessedarrayisindexedbythefullfilenameofeach
includedfileandittracksthisinformationforus.Ifthefileisseenagain,awarningmessageisprinted.Otherwise,the
newfilenameispushedontothestackandprocessingcontinues.
Finally,whengetlineencounterstheendoftheinputfile,thefileisclosedandthestackispopped.Whenstackptris
lessthanzero,theprogramisdone:
stackptr=0
input[stackptr]=ARGV[1]#ARGV[1]isfirstfile
for(;stackptr>=0;stackptr){
while((getline<input[stackptr])>0){
if(tolower($1)!="@include"){
print
continue
}
fpath=pathto($2)
if(fpath==""){
printf("igawk:%s:%d:cannotfind%s\n",
input[stackptr],FNR,$2)>"/dev/stderr"
continue
}
if(!(fpathinprocessed)){
processed[fpath]=input[stackptr]
input[++stackptr]=fpath#pushontostack
}else
print$2,"includedin",input[stackptr],
"alreadyincludedin",
processed[fpath]>"/dev/stderr"
}
close(input[stackptr])
}
}'#closequoteendsèxpand_prog'variable
processed_program=$(gawk"$expand_prog"/dev/stdin<<EOF
$program
EOF
)
Theshellconstructcommand<<markeriscalledaheredocument.Everythingintheshellscriptuptothemarkerisfed
tocommandasinput.Theshellprocessesthecontentsoftheheredocumentforvariableandcommandsubstitution(and
possiblyotherthingsaswell,dependingupontheshell).
Theshellconstruct$()iscalledcommandsubstitution.Theoutputofthecommandinsidetheparenthesesis
substitutedintothecommandline.Becausetheresultisusedinavariableassignment,itissavedasasinglestring,even
iftheresultscontainwhitespace.
251/479
29/7/2016
TheGNUAwkUsersGuide
Theexpandedprogramissavedinthevariableprocessed_program.Itsdoneinthesesteps:
1.Rungawkwiththe@includeprocessingprogram(thevalueoftheexpand_progshellvariable)readingstandard
input.
2.Standardinputisthecontentsoftheusersprogram,fromtheshellvariableprogram.Feeditscontentstogawkvia
aheredocument.
3.Savetheresultsofthisprocessingintheshellvariableprocessed_programbyusingcommandsubstitution.
Thelaststepistocallgawkwiththeexpandedprogram,alongwiththeoriginaloptionsandcommandlinearguments
thattheusersupplied:
evalgawk$opts'"$processed_program"''"$@"'
Theevalcommandisashellconstructthatrerunstheshellsparsingprocess.Thiskeepsthingsproperlyquoted.
Thisversionofigawkrepresentsthefifthversionofthisprogram.Therearefourkeysimplificationsthatmakethe
programworkbetter:
Using@includeevenforthefilesnamedwithfmakesbuildingtheinitialcollectedawkprogrammuchsimpler
allthe@includeprocessingcanbedoneonce.
Nottryingtosavethelinereadwithgetlineinthepathto()functionwhentestingforthefilesaccessibilityfor
usewiththemainprogramsimplifiesthingsconsiderably.
UsingagetlineloopintheBEGINruledoesitallinoneplace.Itisnotnecessarytocallouttoaseparateloopfor
processingnested@includestatements.
Insteadofsavingtheexpandedprograminatemporaryfile,puttingitinashellvariableavoidssomepotential
securityproblems.Thishasthedisadvantagethatthescriptreliesuponmorefeaturesoftheshlanguage,making
ithardertofollowforthosewhoarentfamiliarwithsh.
Also,thisprogramillustratesthatitisoftenworthwhiletocombineshandawkprogrammingtogether.Youcanusually
accomplishquitealot,withouthavingtoresorttolowlevelprogramminginCorC++,anditisfrequentlyeasiertodo
certainkindsofstringandargumentmanipulationusingtheshellthanitisinawk.
Finally,igawkshowsthatitisnotalwaysnecessarytoaddnewfeaturestoaprogramtheycanoftenbelayeredon
top.79
Next:SignatureProgram,Previous:IgawkProgram,Up:MiscellaneousPrograms[Contents][Index]
11.3.10FindingAnagramsfromaDictionary
Aninterestingprogrammingchallengeistosearchforanagramsinawordlist(suchas/usr/share/dict/wordsonmany
GNU/Linuxsystems).Onewordisananagramofanotherifbothwordscontainthesameletters(e.g.,babblingand
blabbing).
Column2,ProblemC,ofJonBentleysProgrammingPearls,SecondEdition,presentsanelegantalgorithm.Theideais
togivewordsthatareanagramsacommonsignature,sortallthewordstogetherbytheirsignatures,andthenprintthem.
Dr.Bentleyobservesthattakingthelettersineachwordandsortingthemproducesthosecommonsignatures.
Thefollowingprogramusesarraysofarraystobringtogetherwordswiththesamesignatureandarraysortingtoprint
thewordsinsortedorder:
#anagram.awkAnimplementationoftheanagramfindingalgorithm
#fromJonBentley's"ProgrammingPearls,"2ndedition.
#AddisonWesley,2000,ISBN0201657880.
#Column2,ProblemC,section2.8,pp1820.
/'s$/{next}#Skippossessives
Theprogramstartswithaheader,andthenaruletoskippossessivesinthedictionaryfile.Thenextrulebuildsupthe
252/479
29/7/2016
TheGNUAwkUsersGuide
datastructure.Thefirstdimensionofthearrayisindexedbythesignaturetheseconddimensionistheworditself:
{
key=word2key($1)#Buildsignature
data[key][$1]=$1#Storewordwithsignature
}
Theword2key()functioncreatesthesignature.Itsplitsthewordapartintoindividualletters,sortstheletters,andthen
joinsthembacktogether:
#word2keysplitwordapartintoletters,sort,andjoinbacktogether
functionword2key(word,a,i,n,result)
{
n=split(word,a,"")
asort(a)
for(i=1;i<=n;i++)
result=resulta[i]
returnresult
}
Finally,theENDruletraversesthearrayandprintsouttheanagramlists.Itsendstheoutputtothesystemsortcommand
becauseotherwisetheanagramswouldappearinarbitraryorder:
END{
sort="sort"
for(keyindata){
#Sortwordswithsamekey
nwords=asorti(data[key],words)
if(nwords==1)
continue
#Andprint.Minorglitch:trailingspaceatendofeachline
for(j=1;j<=nwords;j++)
printf("%s",words[j])|sort
print""|sort
}
close(sort)
}
Hereissomepartialoutputwhentheprogramisrun:
$gawkfanagram.awk/usr/share/dict/words|grep'^b'
babbledblabbed
babblerblabberbrabble
babblersblabbersbrabbles
babblingblabbing
babblyblabby
babelbable
babelsbeslab
baberyyabber
Previous:AnagramProgram,Up:MiscellaneousPrograms[Contents][Index]
11.3.11AndNowforSomethingCompletelyDifferent
ThefollowingprogramwaswrittenbyDavideBriniandispublishedonhiswebsite.Itservesashissignatureinthe
Usenetgroupcomp.lang.awk.Hesuppliesthefollowingcopyrightterms:
Copyright2008DavideBrini
253/479
29/7/2016
TheGNUAwkUsersGuide
Copyinganddistributionofthecodepublishedinthispage,withorwithoutmodification,arepermittedin
anymediumwithoutroyaltyprovidedthecopyrightnoticeandthisnoticearepreserved.
Hereistheprogram:
awk'BEGIN{O="~"~"~";o="=="=="==";o+=+o;x=O""O;while(X++<=x+o+o)c=c"%c";
printfc,(xO)*(xO),x*(xo)o,x*(xO)+xOo,+x*(xO)x+o,X*(o*o+O)+xO,
X*(Xx)o*o,(x+X)*o*o+o,x*(Xx)OO,xO+(O+o+X+x)*(o+O),X*XX*(xO)x+O,
O+X*(o*(o+O)+O),+x+O+X*o,x*(xo),(o+X+x)*o*o(xOO),O+(Xx)*(X+O),xO}'
Weleaveittoyoutodeterminewhattheprogramdoes.(Ifyouaretrulydesperatetounderstandit,seeChrisJohansens
explanation,whichisembeddedintheTexinfosourcefileforthisWebpage.)
Next:ProgramsExercises,Previous:MiscellaneousPrograms,Up:SamplePrograms[Contents][Index]
11.4Summary
Theprogramsprovidedinthischaptercontinueonthethemethatreadingprogramsisanexcellentwaytolearn
GoodProgramming.
Using#!tomakeawkprogramsdirectlyrunnablemakesthemeasiertouse.Otherwise,invoketheprogram
usingawkf.
ReimplementingstandardPOSIXprogramsinawkisapleasantexerciseawksexpressivepowerletsyouwrite
suchprogramsinrelativelyfewlinesofcode,yettheyarefunctionallycompleteandusable.
Oneofstandardawksweaknessesisworkingwithindividualcharacters.Theabilitytousesplit()withthe
emptystringastheseparatorcanconsiderablysimplifysuchtasks.
TheexamplesheredemonstratetheusefulnessofthelibraryfunctionsfromLibraryFunctions,foranumberof
real(ifsmall)programs.
BesidesreinventingPOSIXwheels,otherprogramssolvedaselectionofinterestingproblems,suchasfinding
duplicatewordsintext,printingmailinglabels,andfindinganagrams.
Previous:ProgramsSummary,Up:SamplePrograms[Contents][Index]
11.5Exercises
1.Rewritecut.awk(seeCutProgram)usingsplit()with""astheseparator.
2.InEgrepProgram,wementionedthategrepicouldbesimulatedinversionsofawkwithoutIGNORECASEby
usingtolower()onthelineandthepattern.Inafootnotethere,wealsomentionedthatthissolutionhasabug:the
translatedlineisoutput,andnottheoriginalone.Fixthisproblem.
3.ThePOSIXversionofidtakesoptionsthatcontrolwhichinformationisprinted.Modifytheawkversion(seeId
Program)toacceptthesameargumentsandperforminthesameway.
4.Thesplit.awkprogram(seeSplitProgram)assumesthatlettersarecontiguousinthecharacterset,whichisnt
trueforEBCDICsystems.Fixthisproblem.(Hint:Consideradifferentwaytoworkthroughthealphabet,
withoutrelyingonord()andchr().)
5.Inuniq.awk(seeUniqProgram,thelogicforchoosingwhichlinestoprintrepresentsastatemachine,whichisa
devicethatcanbeinoneofasetnumberofstableconditionsdependingonitspreviousconditionandonthe
presentvaluesofitsinputs.80BrianKernighansuggeststhatanalternativeapproachtostatemachinesistojust
readtheinputintoanarray,thenuseindexing.Itsalmostalwayseasiercode,andformostinputswhereyou
wouldusethis,justasfast.Rewritethelogictofollowthissuggestion.
6.Whycantthewc.awkprogram(seeWcProgram)justusethevalueofFNRinendfile()?Hint:Examinethecode
inFiletransFunction.
7.Manipulationofindividualcharactersinthetranslateprogram(seeTranslateProgram)ispainfulusingstandard
awkfunctions.Giventhatgawkcansplitstringsintoindividualcharactersusing""astheseparator,howmightyou
usethisfeaturetosimplifytheprogram?
8.Theextract.awkprogram(seeExtractProgram)waswrittenbeforegawkhadthegensub()function.Useitto
simplifythecode.
9.Comparetheperformanceoftheawksed.awkprogram(seeSimpleSed)withthemorestraightforward:
254/479
29/7/2016
TheGNUAwkUsersGuide
BEGIN{
pat=ARGV[1]
repl=ARGV[2]
ARGV[1]=ARGV[2]=""
}
{gsub(pat,repl);print}
10.Whataretheadvantagesanddisadvantagesofawksed.awkversustherealsedutility?
11.InIgawkProgram,wementionedthatnottryingtosavethelinereadwithgetlineinthepathto()functionwhen
testingforthefilesaccessibilityforusewiththemainprogramsimplifiesthingsconsiderably.Whatproblem
doesthisengenderthough?
12.Asanadditionalexampleoftheideathatitisnotalwaysnecessarytoaddnewfeaturestoaprogram,consider
theideaofhavingtwofilesinadirectoryinthesearchpath:
default.awk
Thisfilecontainsasetofdefaultlibraryfunctions,suchasgetopt()andassert().
site.awk
Thisfilecontainslibraryfunctionsthatarespecifictoasiteorinstallationi.e.,locallydevelopedfunctions.
Havingaseparatefileallowsdefault.awktochangewithnewgawkreleases,withoutrequiringthesystem
administratortoupdateiteachtimebyaddingthelocalfunctions.
Oneusersuggestedthatgawkbemodifiedtoautomaticallyreadthesefilesuponstartup.Instead,itwouldbevery
simpletomodifyigawktodothis.Sinceigawkcanprocessnested@includedirectives,default.awkcouldsimply
contain@includestatementsforthedesiredlibraryfunctions.Makethischange.
13.Modifyanagram.awk(seeAnagramProgram),toavoidtheuseoftheexternalsortutility.
Next:Internationalization,Previous:SamplePrograms,Up:Top[Contents][Index]
PartIII:
Writedocumentationasifwhoeverreadsitisaviolentpsychopathwhoknowswhereyoulive.
SteveEnglish,asquotedbyPeterLangston
Thischapterdiscussesadvancedfeaturesingawk.Itsabitofagrabbagofitemsthatareotherwiseunrelatedtoeach
other.First,welookatacommandlineoptionthatallowsgawktorecognizenondecimalnumbersininputdata,notjust
inawkprograms.Then,gawksspecialfeaturesforsortingarraysarepresented.Next,twowayI/O,discussedbrieflyin
earlierpartsofthisWebpage,isdescribedinfulldetail,alongwiththebasicsofTCP/IPnetworking.Finally,wesee
howgawkcanprofileanawkprogram,makingitpossibletotuneitforperformance.
Additionaladvancedfeaturesarediscussedinseparatechaptersoftheirown:
Internationalization,discusseshowtointernationalizeyourawkprograms,sothattheycanspeakmultiplenational
languages.
Debugger,describesgawksbuiltincommandlinedebuggerfordebuggingawkprograms.
ArbitraryPrecisionArithmetic,describeshowyoucanusegawktoperformarbitraryprecisionarithmetic.
DynamicExtensions,discussestheabilitytodynamicallyaddnewbuiltinfunctionstogawk.
255/479
29/7/2016
TheGNUAwkUsersGuide
NondecimalData:
Allowingnondecimalinputdata.
ArraySorting:
Facilitiesforcontrollingarraytraversalandsortingarrays.
TwowayI/O:
Twowaycommunicationswithanotherprocess.
TCP/IPNetworking:
Usinggawkfornetworkprogramming.
Profiling:
Profilingyourawkprograms.
AdvancedFeaturesSummary: Summaryofadvancedfeatures.
Next:ArraySorting,Up:AdvancedFeatures[Contents][Index]
12.1AllowingNondecimalInputData
Ifyourungawkwiththenondecimaldataoption,youcanhavenondecimalvaluesinyourinputdata:
$echo01231230x123|
>gawknondecimaldata'{printf"%d,%d,%d\n",$1,$2,$3}'
|83,123,291
Forthisfeaturetowork,writeyourprogramsothatgawktreatsyourdataasnumeric:
$echo01231230x123|gawk'{print$1,$2,$3}'
|01231230x123
Theprintstatementtreatsitsexpressionsasstrings.Althoughthefieldscanactasnumberswhennecessary,theyare
stillstrings,soprintdoesnottrytotreatthemnumerically.Youneedtoaddzerotoafieldtoforceittobetreatedasa
number.Forexample:
$echo01231230x123|gawknondecimaldata'
>{print$1,$2,$3
>print$1+0,$2+0,$3+0}'
|01231230x123
|83123291
Becauseitiscommontohavedecimaldatawithleadingzeros,andbecauseusingthisfacilitycouldleadtosurprising
results,thedefaultistoleaveitdisabled.Ifyouwantit,youmustexplicitlyrequestit.
CAUTION:Useofthisoptionisnotrecommended.Itcanbreakoldprogramsverybadly.Instead,usethe
strtonum()functiontoconvertyourdata(seeStringFunctions).Thismakesyourprogramseasiertowrite
andeasiertoread,andleadstolesssurprisingresults.
Thisoptionmaydisappearinafutureversionofgawk.
Next:TwowayI/O,Previous:NondecimalData,Up:AdvancedFeatures[Contents][Index]
12.2ControllingArrayTraversalandArraySorting
gawkletsyoucontroltheorderinwhichafor(indxinarray)looptraversesanarray.
Inaddition,twobuiltinfunctions,asort()andasorti(),letyousortarraysbasedonthearrayvaluesandindices,
respectively.Thesetwofunctionsalsoprovidecontroloverthesortingcriteriausedtoordertheelementsduringsorting.
ControllingArrayTraversal: HowtousePROCINFO["sorted_in"].
ArraySortingFunctions:
Howtouseasort()andasorti().
Next:ArraySortingFunctions,Up:ArraySorting[Contents][Index]
12.2.1ControllingArrayTraversal
256/479
29/7/2016
TheGNUAwkUsersGuide
Bydefault,theorderinwhichafor(indxinarray)loopscansanarrayisnotdefineditisgenerallybaseduponthe
internalimplementationofarraysinsideawk.
Often,though,itisdesirabletobeabletoloopovertheelementsinaparticularorderthatyou,theprogrammer,choose.
gawkletsyoudothis.
ControllingScanning,describeshowyoucanassignspecial,predefinedvaluestoPROCINFO["sorted_in"]inorderto
controltheorderinwhichgawktraversesanarrayduringaforloop.
Inaddition,thevalueofPROCINFO["sorted_in"]canbeafunctionname.81Thisletsyoutraverseanarraybasedonany
customcriterion.Thearrayelementsareorderedaccordingtothereturnvalueofthisfunction.Thecomparisonfunction
shouldbedefinedwithatleastfourarguments:
functioncomp_func(i1,v1,i2,v2)
{
compareelements1and2insomefashion
return<0;0;or>0
}
Here,i1andi2aretheindices,andv1andv2arethecorrespondingvaluesofthetwoelementsbeingcompared.Either
v1orv2,orboth,canbearraysifthearraybeingtraversedcontainssubarraysasvalues.(SeeArraysofArrays,formore
informationaboutsubarrays.)Thethreepossiblereturnvaluesareinterpretedasfollows:
comp_func(i1,v1,i2,v2)<0
Indexi1comesbeforeindexi2duringlooptraversal.
comp_func(i1,v1,i2,v2)==0
Indicesi1andi2cometogether,buttherelativeorderwithrespecttoeachotherisundefined.
comp_func(i1,v1,i2,v2)>0
Indexi1comesafterindexi2duringlooptraversal.
Ourfirstcomparisonfunctioncanbeusedtoscananarrayinnumericalorderoftheindices:
functioncmp_num_idx(i1,v1,i2,v2)
{
#numericalindexcomparison,ascendingorder
return(i1i2)
}
Oursecondfunctiontraversesanarraybasedonthestringorderoftheelementvaluesratherthanbyindices:
functioncmp_str_val(i1,v1,i2,v2)
{
#stringvaluecomparison,ascendingorder
v1=v1""
v2=v2""
if(v1<v2)
return1
return(v1!=v2)
}
Thethirdcomparisonfunctionmakesallnumbers,andnumericstringswithoutanyleadingortrailingspaces,comeout
firstduringlooptraversal:
functioncmp_num_str_val(i1,v1,i2,v2,n1,n2)
{
#numbersbeforestringvaluecomparison,ascendingorder
n1=v1+0
n2=v2+0
if(n1==v1)
257/479
29/7/2016
TheGNUAwkUsersGuide
return(n2==v2)?(n1n2):1
elseif(n2==v2)
return1
return(v1<v2)?1:(v1!=v2)
}
Hereisamainprogramtodemonstratehowgawkbehavesusingeachofthepreviousfunctions:
BEGIN{
data["one"]=10
data["two"]=20
data[10]="one"
data[100]=100
data[20]="two"
f[1]="cmp_num_idx"
f[2]="cmp_str_val"
f[3]="cmp_num_str_val"
for(i=1;i<=3;i++){
printf("Sortfunction:%s\n",f[i])
PROCINFO["sorted_in"]=f[i]
for(jindata)
printf("\tdata[%s]=%s\n",j,data[j])
print""
}
}
Herearetheresultswhentheprogramisrun:
$gawkfcompdemo.awk
|Sortfunction:cmp_num_idxSortbynumericindex
|data[two]=20
|data[one]=10Bothstringsarenumericallyzero
|data[10]=one
|data[20]=two
|data[100]=100
|
|Sortfunction:cmp_str_valSortbyelementvaluesasstrings
|data[one]=10
|data[100]=100String100islessthanstring20
|data[two]=20
|data[10]=one
|data[20]=two
|
|Sortfunction:cmp_num_str_valSortallnumericvaluesbeforeallstrings
|data[one]=10
|data[two]=20
|data[100]=100
|data[10]=one
|data[20]=two
ConsidersortingtheentriesofaGNU/Linuxsystempasswordfileaccordingtologinname.Thefollowingprogram
sortsrecordsbyaspecificfieldpositionandcanbeusedforthispurpose:
#passwdsort.awksimpleprogramtosortbyfieldposition
#fieldpositionisspecifiedbytheglobalvariablePOS
functioncmp_field(i1,v1,i2,v2)
{
#comparisonbyvalue,asstring,andascendingorder
returnv1[POS]<v2[POS]?1:(v1[POS]!=v2[POS])
}
{
for(i=1;i<=NF;i++)
a[NR][i]=$i
}
258/479
29/7/2016
TheGNUAwkUsersGuide
END{
PROCINFO["sorted_in"]="cmp_field"
if(POS<1||POS>NF)
POS=1
for(iina){
for(j=1;j<=NF;j++)
printf("%s%c",a[i][j],j<NF?":":"")
print""
}
}
Thefirstfieldineachentryofthepasswordfileistheusersloginname,andthefieldsareseparatedbycolons.Each
recorddefinesasubarray,witheachfieldasanelementinthesubarray.Runningtheprogramproducesthefollowing
output:
$gawkvPOS=1F:fsort.awk/etc/passwd
|adm:x:3:4:adm:/var/adm:/sbin/nologin
|apache:x:48:48:Apache:/var/www:/sbin/nologin
|avahi:x:70:70:Avahidaemon:/:/sbin/nologin
Thecomparisonshouldnormallyalwaysreturnthesamevaluewhengivenaspecificpairofarrayelementsasits
arguments.Ifinconsistentresultsarereturned,thentheorderisundefined.Thisbehaviorcanbeexploitedtointroduce
randomorderintootherwiseseeminglyordereddata:
functioncmp_randomize(i1,v1,i2,v2)
{
#randomorder(caution:thismayneverterminate!)
return(24*rand())
}
Asalreadymentioned,theorderoftheindicesisarbitraryiftwoelementscompareequal.Thisisusuallynotaproblem,
butlettingthetiedelementscomeoutinarbitraryordercanbeanissue,especiallywhencomparingitemvalues.The
partialorderingoftheequalelementsmaychangethenexttimethearrayistraversed,ifotherelementsareaddedtoor
removedfromthearray.Onewaytoresolvetieswhencomparingelementswithotherwiseequalvaluesistoincludethe
indicesinthecomparisonrules.Notethatdoingthismaymakethelooptraversallessefficient,soconsideritonlyif
necessary.Thefollowingcomparisonfunctionsforceadeterministicorder,andarebasedonthefactthatthe(string)
indicesoftwoelementsareneverequal:
functioncmp_numeric(i1,v1,i2,v2)
{
#numericalvalue(andindex)comparison,descendingorder
return(v1!=v2)?(v2v1):(i2i1)
}
functioncmp_string(i1,v1,i2,v2)
{
#stringvalue(andindex)comparison,descendingorder
v1=v1i1
v2=v2i2
return(v1>v2)?1:(v1!=v2)
}
Acustomcomparisonfunctioncanoftensimplifyorderedlooptraversal,andtheskyisreallythelimitwhenitcomesto
designingsuchafunction.
Whenstringcomparisonsaremadeduringasort,eitherforelementvalueswhereoneorbotharentnumbers,orfor
elementindiceshandledasstrings,thevalueofIGNORECASE(seeBuiltinVariables)controlswhetherthecomparisons
treatcorrespondingupperandlowercaselettersasequivalentordistinct.
Anotherpointtokeepinmindisthatinthecaseofsubarrays,theelementvaluescanthemselvesbearraysaproduction
comparisonfunctionshouldusetheisarray()function(seeTypeFunctions)tocheckforthis,andchooseadefined
sortingorderforsubarrays.
259/479
29/7/2016
TheGNUAwkUsersGuide
AllsortingbasedonPROCINFO["sorted_in"]isdisabledinPOSIXmode,becausethePROCINFOarrayisnotspecialin
thatcase.
Asasidenote,sortingthearrayindicesbeforetraversingthearrayhasbeenreportedtoadda15%to20%overheadto
theexecutiontimeofawkprograms.Forthisreason,sortedarraytraversalisnotthedefault.
Previous:ControllingArrayTraversal,Up:ArraySorting[Contents][Index]
12.2.2SortingArrayValuesandIndiceswithgawk
Inmostawkimplementations,sortinganarrayrequireswritingasort()function.Thiscanbeeducationalforexploring
differentsortingalgorithms,butusuallythatsnotthepointoftheprogram.gawkprovidesthebuiltinasort()and
asorti()functions(seeStringFunctions)forsortingarrays.Forexample:
populatethearraydata
n=asort(data)
for(i=1;i<=n;i++)
dosomethingwithdata[i]
Afterthecalltoasort(),thearraydataisindexedfrom1tosomenumbern,thetotalnumberofelementsindata.(This
countisasort()sreturnvalue.)data[1]<=data[2]<=data[3],andsoon.Thedefaultcomparisonisbasedonthetype
oftheelements(seeTypingandComparison).Allnumericvaluescomebeforeallstringvalues,whichinturncome
beforeallsubarrays.
Animportantsideeffectofcallingasort()isthatthearraysoriginalindicesareirrevocablylost.Asthisisntalways
desirable,asort()acceptsasecondargument:
populatethearraysource
n=asort(source,dest)
for(i=1;i<=n;i++)
dosomethingwithdest[i]
Inthiscase,gawkcopiesthesourcearrayintothedestarrayandthensortsdest,destroyingitsindices.However,the
sourcearrayisnotaffected.
Often,whatsneededistosortonthevaluesoftheindicesinsteadofthevaluesoftheelements.Todothat,usethe
asorti()function.Theinterfaceandbehaviorareidenticaltothatofasort(),exceptthattheindexvaluesareusedfor
sortingandbecomethevaluesoftheresultarray:
{source[$0]=some_func($0)}
END{
n=asorti(source,dest)
for(i=1;i<=n;i++){
Workwithsortedindicesdirectly:
dosomethingwithdest[i]
Accessoriginalarrayviasortedindices:
dosomethingwithsource[dest[i]]
}
}
Sofar,sogood.Nowitstartstogetinteresting.Bothasort()andasorti()acceptathirdstringargumenttocontrol
comparisonofarrayelements.Whenweintroducedasort()andasorti()inStringFunctions,weignoredthisthird
argumenthowever,nowisthetimetodescribehowthisargumentaffectsthesetwofunctions.
Basically,thethirdargumentspecifieshowthearrayistobesorted.Therearetwopossibilities.Aswith
PROCINFO["sorted_in"],thisargumentmaybeoneofthepredefinednamesthatgawkprovides(seeControlling
Scanning),oritmaybethenameofauserdefinedfunction(seeControllingArrayTraversal).
Inthelattercase,thefunctioncancompareelementsinanywayitchooses,takingintoaccountjusttheindices,justthe
260/479
29/7/2016
TheGNUAwkUsersGuide
values,orboth.Thisisextremelypowerful.
Oncethearrayissorted,asort()takesthevaluesintheirfinalorderandusesthemtofillintheresultarray,whereas
asorti()takestheindicesintheirfinalorderandusesthemtofillintheresultarray.
NOTE:Copyingarrayindicesandelementsisntexpensiveintermsofmemory.Internally,gawkmaintains
referencecountstodata.Forexample,whenasort()copiesthefirstarraytothesecondone,thereisonly
onecopyoftheoriginalarrayelementsdata,eventhoughbotharraysusethevalues.
BecauseIGNORECASEaffectsstringcomparisons,thevalueofIGNORECASEalsoaffectssortingforbothasort()and
asorti().Notealsothatthelocalessortingorderdoesnotcomeintoplaycomparisonsarebasedoncharactervalues
only.82
Next:TCP/IPNetworking,Previous:ArraySorting,Up:AdvancedFeatures[Contents][Index]
12.3TwoWayCommunicationswithAnotherProcess
Itisoftenusefultobeabletosenddatatoaseparateprogramforprocessingandthenreadtheresult.Thiscanalwaysbe
donewithtemporaryfiles:
#Writethedataforprocessing
tempfile=("mydata."PROCINFO["pid"])
while(notdonewithdata)
printdata|("subprogram>"tempfile)
close("subprogram>"tempfile)
#Readtheresults,removetempfilewhendone
while((getlinenewdata<tempfile)>0)
processnewdataappropriately
close(tempfile)
system("rm"tempfile)
Thisworks,butnotelegantly.Amongotherthings,itrequiresthattheprogramberuninadirectorythatcannotbe
sharedamongusersforexample,/tmpwillnotdo,asanotherusermighthappentobeusingatemporaryfilewiththe
samename.83
However,withgawk,itispossibletoopenatwowaypipetoanotherprocess.Thesecondprocessistermedacoprocess,
asitrunsinparallelwithgawk.Thetwowayconnectioniscreatedusingthe|&operator(borrowedfromtheKorn
shell,ksh):84
do{
printdata|&"subprogram"
"subprogram"|&getlineresults
}while(datalefttoprocess)
close("subprogram")
ThefirsttimeanI/Ooperationisexecutedusingthe|&operator, gawkcreatesatwowaypipelinetoachildprocess
thatrunstheotherprogram.Outputcreatedwithprintorprintfiswrittentotheprogramsstandardinput,andoutput
fromtheprogramsstandardoutputcanbereadbythegawkprogramusinggetline.Asisthecasewithprocessesstarted
by|,thesubprogramcanbeanyprogram,orpipelineofprograms,thatcanbestartedbytheshell.
Therearesomecautionaryitemstobeawareof:
Asthecodeinsidegawkcurrentlystands,thecoprocesssstandarderrorgoestothesameplacethattheparent
gawksstandarderrorgoes.Itisnotpossibletoreadthechildsstandarderrorseparately.
I/Obufferingmaybeaproblem.gawkautomaticallyflushesalloutputdownthepipetothecoprocess.However,
ifthecoprocessdoesnotflushitsoutput,gawkmayhangwhendoingagetlineinordertoreadthecoprocesss
results.Thiscouldleadtoasituationknownasdeadlock,whereeachprocessiswaitingfortheotheronetodo
something.
261/479
29/7/2016
TheGNUAwkUsersGuide
Itispossibletoclosejustoneendofthetwowaypipetoacoprocess,bysupplyingasecondargumenttotheclose()
functionofeither"to"or"from"(seeCloseFilesAndPipes).Thesestringstellgawktoclosetheendofthepipethat
sendsdatatothecoprocessortheendthatreadsfromit,respectively.
Thisisparticularlynecessaryinordertousethesystemsortutilityaspartofacoprocesssortmustreadallofitsinput
databeforeitcanproduceanyoutput.Thesortprogramdoesnotreceiveanendoffileindicationuntilgawkclosesthe
writeendofthepipe.
Whenyouhavefinishedwritingdatatothesortutility,youcanclosethe"to"endofthepipe,andthenstartreading
sorteddataviagetline.Forexample:
BEGIN{
command="LC_ALL=Csort"
n=split("abcdefghijklmnopqrstuvwxyz",a,"")
for(i=n;i>0;i)
printa[i]|&command
close(command,"to")
while((command|&getlineline)>0)
print"got",line
close(command)
}
Thisprogramwritesthelettersofthealphabetinreverseorder,oneperline,downthetwowaypipetosort.Itthen
closesthewriteendofthepipe,sothatsortreceivesanendoffileindication.Thiscausessorttosortthedataand
writethesorteddatabacktothegawkprogram.Onceallofthedatahasbeenread,gawkterminatesthecoprocessand
exits.
Asasidenote,theassignmentLC_ALL=CinthesortcommandensurestraditionalUnix(ASCII)sortingfromsort.This
isnotstrictlynecessaryhere,butitsgoodtoknowhowtodothis.
Youmayalsousepseudottys(ptys)fortwowaycommunicationinsteadofpipes,ifyoursystemsupportsthem.Thisis
doneonapercommandbasis,bysettingaspecialelementinthePROCINFOarray(seeAutoset),likeso:
command="sortnr"#command,saveinconveniencevariable
PROCINFO[command,"pty"]=1#updatePROCINFO
print|&command#starttwowaypipe
Usingptysusuallyavoidsthebufferdeadlockissuesdescribedearlier,atsomelossinperformance.Ifyoursystemdoes
nothaveptys,orifallthesystemsptysareinuse,gawkautomaticallyfallsbacktousingregularpipes.
Next:Profiling,Previous:TwowayI/O,Up:AdvancedFeatures[Contents][Index]
12.4UsinggawkforNetworkProgramming
EMRED:
Ahostisahostfromcoasttocoast,
andnobodytalkstoahostthatsclose,
unlessthehostthatisntclose
isbusy,hung,ordead.
MikeOBrien(akaMr.Protocol)
Inadditiontobeingabletoopenatwowaypipelinetoacoprocessonthesamesystem(seeTwowayI/O),itispossible
tomakeatwowayconnectiontoanotherprocessonanothersystemacrossanIPnetworkconnection.
Youcanthinkofthisasjustaverylongtwowaypipelinetoacoprocess.Thewaygawkdecidesthatyouwanttouse
TCP/IPnetworkingisbyrecognizingspecialfilenamesthatbeginwithoneof/inet/,/inet4/,or/inet6/.
262/479
29/7/2016
TheGNUAwkUsersGuide
Thefullsyntaxofthespecialfilenameis/nettype/protocol/localport/remotehost/remoteport.Thecomponents
are:
nettype
SpecifiesthekindofInternetconnectiontomake.Use/inet4/toforceIPv4,and/inet6/toforceIPv6.Plain
/inet/(whichusedtobetheonlyoption)usesthesystemdefault,mostlikelyIPv4.
protocol
TheprotocoltouseoverIP.Thismustbeeithertcp,orudp,foraTCPorUDPIPconnection,respectively.
TCPshouldbeusedformostapplications.
localport
ThelocalTCPorUDPportnumbertouse.Useaportnumberof0whenyouwantthesystemtopickaport.
ThisiswhatyoushoulddowhenwritingaTCPorUDPclient.Youmayalsouseawellknownservicename,
suchassmtpor http,inwhichcasegawkattemptstodeterminethepredefinedportnumberusingtheC
getaddrinfo()function.
remotehost
TheIPaddressorfullyqualifieddomainnameoftheInternethosttowhichyouwanttoconnect.
remoteport
TheTCPorUDPportnumbertouseonthegivenremotehost.Again,use0ifyoudontcare,orelseawell
knownservicename.
NOTE:Failureinopeningatwowaysocketwillresultinanonfatalerrorbeingreturnedtothecalling
code.ThevalueofERRNOindicatestheerror(seeAutoset).
Considerthefollowingverysimpleexample:
BEGIN{
Service="/inet/tcp/0/localhost/daytime"
Service|&getline
print$0
close(Service)
}
ThisprogramreadsthecurrentdateandtimefromthelocalsystemsTCPdaytimeserver.Itthenprintstheresultsand
closestheconnection.
Becausethistopicisextensive,theuseofgawkforTCP/IPprogrammingisdocumentedseparately.SeeTCP/IP
Internetworkingwithgawk,whichcomesaspartofthegawkdistribution,foramuchmorecompleteintroductionand
discussion,aswellasextensiveexamples.
Next:AdvancedFeaturesSummary,Previous:TCP/IPNetworking,Up:AdvancedFeatures[Contents][Index]
12.5ProfilingYourawkPrograms
Youmayproduceexecutiontracesofyourawkprograms.Thisisdonebypassingtheoptionprofiletogawk.When
gawkhasfinishedrunning,itcreatesaprofileofyourprograminafilenamedawkprof.out.Becauseitisprofiling,it
alsoexecutesupto45%slowerthangawknormallydoes.
Asshowninthefollowingexample,theprofileoptioncanbeusedtochangethenameofthefilewheregawkwill
writetheprofile:
263/479
29/7/2016
TheGNUAwkUsersGuide
gawkprofile=myprog.proffmyprog.awkdata1data2
Intheprecedingexample,gawkplacestheprofileinmyprog.profinsteadofinawkprof.out.
Hereisasamplesessionshowingasimpleawkprogram,itsinputdata,andtheresultsfromrunninggawkwiththe
profileoption.First,theawkprogram:
BEGIN{print"FirstBEGINrule"}
END{print"FirstENDrule"}
/foo/{
print"matched/foo/,gosh"
for(i=1;i<=3;i++)
sing()
}
{
if(/foo/)
print"ifistrue"
else
print"elseistrue"
}
BEGIN{print"SecondBEGINrule"}
END{print"SecondENDrule"}
functionsing(dummy)
{
print"Igottabeme!"
}
Followingistheinputdata:
foo
bar
baz
foo
junk
Hereistheawkprof.outthatresultsfromrunningthegawkprofileronthisprogramanddata(thisexamplealso
illustratesthatawkprogrammerssometimesgetupveryearlyinthemorningtowork):
#gawkprofile,createdMonSep2905:16:212014
#BEGINrule(s)
BEGIN{
1print"FirstBEGINrule"
}
BEGIN{
1print"SecondBEGINrule"
}
#Rule(s)
5/foo/{#2
2print"matched/foo/,gosh"
6for(i=1;i<=3;i++){
6sing()
}
}
5{
5if(/foo/){#2
264/479
29/7/2016
TheGNUAwkUsersGuide
2print"ifistrue"
3}else{
3print"elseistrue"
}
}
#ENDrule(s)
END{
1print"FirstENDrule"
}
END{
1print"SecondENDrule"
}
#Functions,listedalphabetically
6functionsing(dummy)
{
6print"Igottabeme!"
}
Thisexampleillustratesmanyofthebasicfeaturesofprofilingoutput.Theyareasfollows:
TheprogramisprintedintheorderBEGINrules,BEGINFILErules,patternactionrules,ENDFILErules,ENDrules,and
functions,listedalphabetically.MultipleBEGINandENDrulesretaintheirseparateidentities,asdomultiple
BEGINFILEandENDFILErules.
Patternactionruleshavetwocounts.Thefirstcount,totheleftoftherule,showshowmanytimestherules
patternwastested.Thesecondcount,totherightoftherulesopeningleftbraceinacomment,showshowmany
timestherulesactionwasexecuted.Thedifferencebetweenthetwoindicateshowmanytimestherulespattern
evaluatedtofalse.
Similarly,thecountforanifelsestatementshowshowmanytimestheconditionwastested.Totherightofthe
openingleftbracefortheifsbodyisacountshowinghowmanytimestheconditionwastrue.Thecountforthe
elseindicateshowmanytimesthetestfailed.
Thecountforaloopheader(suchasfororwhile)showshowmanytimesthelooptestwasexecuted.(Because
ofthis,youcantjustlookatthecountonthefirststatementinaruletodeterminehowmanytimestherulewas
executed.Ifthefirststatementisaloop,thecountismisleading.)
Foruserdefinedfunctions,thecountnexttothefunctionkeywordindicateshowmanytimesthefunctionwas
called.Thecountsnexttothestatementsinthebodyshowhowmanytimesthosestatementswereexecuted.
ThelayoutusesK&RstylewithTABs.Bracesareusedeverywhere,evenwhenthebodyofanif,else,or
loopisonlyasinglestatement.
Parenthesesareusedonlywhereneeded,asindicatedbythestructureoftheprogramandtheprecedencerules.
Forexample,(3+5)*4meansaddthreeandfive,thenmultiplythetotalbyfour.However,3+5*4has
noparentheses,andmeans3+(5*4).
Parenthesesareusedaroundtheargumentstoprintandprintfonlywhentheprintorprintfstatementis
followedbyaredirection.Similarly,ifthetargetofaredirectionisntascalar,itgetsparenthesized.
gawksuppliesleadingcommentsinfrontoftheBEGINandENDrules,theBEGINFILEandENDFILErules,thepattern
actionrules,andthefunctions.
Theprofiledversionofyourprogrammaynotlookexactlylikewhatyoutypedwhenyouwroteit.Thisisbecausegawk
createstheprofiledversionbyprettyprintingitsinternalrepresentationoftheprogram.Theadvantagetothisisthat
gawkcanproduceastandardrepresentation.Thedisadvantageisthatallsourcecodecommentsarelost.Also,things
suchas:
/foo/
comeoutas:
/foo/{
print$0
}
265/479
29/7/2016
TheGNUAwkUsersGuide
whichiscorrect,butpossiblyunexpected.
Besidescreatingprofileswhenaprogramhascompleted,gawkcanproduceaprofilewhileitisrunning.Thisisusefulif
yourawkprogramgoesintoaninfiniteloopandyouwanttoseewhathasbeenexecuted.Tousethisfeature,rungawk
withtheprofileoptioninthebackground:
$gawkprofilefmyprog&
[1]13992
TheshellprintsajobnumberandprocessIDnumberinthiscase,13992.UsethekillcommandtosendtheUSR1
signaltogawk:
$killUSR113992
Asusual,theprofiledversionoftheprogramiswrittentoawkprof.out,ortoadifferentfileifonewasspecifiedwiththe
profileoption.
Alongwiththeregularprofile,asshownearlier,theprofilefileincludesatraceofanyactivefunctions:
#FunctionCallStack:
#3.baz
#2.bar
#1.foo
#main
YoumaysendgawktheUSR1signalasmanytimesasyoulike.Eachtime,theprofileandfunctioncalltraceare
appendedtotheoutputprofilefile.
IfyouusetheHUPsignalinsteadoftheUSR1signal,gawkproducestheprofileandthefunctioncalltraceandthenexits.
WhengawkrunsonMSWindowssystems,itusestheINTandQUITsignalsforproducingtheprofile,andinthecaseof
theINTsignal,gawkexits.Thisisbecausethesesystemsdontsupportthekillcommand,sotheonlysignalsyoucan
delivertoaprogramarethosegeneratedbythekeyboard.TheINTsignalisgeneratedbytheCtrlcorCtrlBREAKkey,
whiletheQUITsignalisgeneratedbytheCtrl\key.
Finally,gawkalsoacceptsanotheroption,prettyprint.Whencalledthisway,gawkprettyprintstheprograminto
awkprof.out,withoutanyexecutioncounts.
NOTE:Theprettyprintoptionstillrunsyourprogram.Thiswillchangeinthenextmajorrelease.
Previous:Profiling,Up:AdvancedFeatures[Contents][Index]
12.6Summary
Thenondecimaldataoptioncausesgawktotreatoctalandhexadecimallookinginputdataasoctaland
hexadecimal.Thisoptionshouldbeusedwithcautionornotatalluseofstrtonum()ispreferable.Notethatthis
optionmaydisappearinafutureversionofgawk.
Youcantakeovercompletecontrolofsortinginfor(indxinarray)arraytraversalbysetting
PROCINFO["sorted_in"]tothenameofauserdefinedfunctionthatdoesthecomparisonofarrayelementsbased
onindexandvalue.
Similarly,youcansupplythenameofauserdefinedcomparisonfunctionasthethirdargumenttoeitherasort()
orasorti()tocontrolhowthosefunctionssortarrays.Oryoumayprovideoneofthepredefinedcontrolstrings
thatworkforPROCINFO["sorted_in"].
Youcanusethe|&operatortocreateatwowaypipetoacoprocess.Youreadfromthecoprocesswithgetline
andwritetoitwithprintorprintf.Useclose()tocloseoffthecoprocesscompletely,oroptionally,closeoff
onesideofthetwowaycommunications.
Byusingspecialfilenameswiththe|&operator,youcanopenaTCP/IP(orUDP/IP)connectiontoremote
hostsontheInternet.gawksupportsbothIPv4andIPv6.
266/479
29/7/2016
TheGNUAwkUsersGuide
Youcangeneratestatementcountprofilesofyourprogram.Thiscanhelpyoudeterminewhichpartsofyour
programmaybetakingthemosttimeandletyoutunethemmoreeasily.SendingtheUSR1signalwhileprofiling
causesgawktodumptheprofileandkeepgoing,includingafunctioncallstack.
Youcanalsojustprettyprinttheprogram.Thiscurrentlyalsorunstheprogram,butthatwillchangeinthenext
majorrelease.
Next:Debugger,Previous:AdvancedFeatures,Up:Top[Contents][Index]
Onceuponatime,computermakerswrotesoftwarethatworkedonlyinEnglish.Eventually,hardwareandsoftware
vendorsnoticedthatiftheirsystemsworkedinthenativelanguagesofnonEnglishspeakingcountries,theywereable
tosellmoresystems.Asaresult,internationalizationandlocalizationofprogramsandsoftwaresystemsbecamea
commonpractice.
Formanyyears,theabilitytoprovideinternationalizationwaslargelyrestrictedtoprogramswritteninCandC++.This
chapterdescribestheunderlyinglibrarygawkusesforinternationalization,aswellashowgawkmakes
internationalizationfeaturesavailableattheawkprogramlevel.Havinginternationalizationavailableattheawklevel
givessoftwaredevelopersadditionalflexibilitytheyarenolongerforcedtowriteinCorC++when
internationalizationisarequirement.
I18NandL10N: InternationalizationandLocalization.
Explaininggettext: HowGNUgettextworks.
Programmeri18n: Featuresfortheprogrammer.
Translatori18n: Featuresforthetranslator.
I18NExample:
Asimplei18nexample.
GawkI18N:
gawkisalsointernationalized.
I18NSummary: SummaryofI18Nstuff.
Next:Explaininggettext,Up:Internationalization[Contents][Index]
13.1InternationalizationandLocalization
Internationalizationmeanswriting(ormodifying)aprogramonce,insuchawaythatitcanusemultiplelanguages
withoutrequiringfurthersourcecodechanges.Localizationmeansprovidingthedatanecessaryforaninternationalized
programtoworkinaparticularlanguage.Mosttypically,thesetermsrefertofeaturessuchasthelanguageusedfor
printingerrormessages,thelanguageusedtoreadresponses,andinformationrelatedtohownumericalandmonetary
valuesareprintedandread.
Next:Programmeri18n,Previous:I18NandL10N,Up:Internationalization[Contents][Index]
13.2GNUgettext
gawkusesGNUgettexttoprovideitsinternationalizationfeatures.ThefacilitiesinGNUgettextfocusonmessages:
stringsprintedbyaprogram,eitherdirectlyorviaformattingwithprintforsprintf().85
WhenusingGNUgettext,eachapplicationhasitsowntextdomain.Thisisauniquename,suchaskpilotorgawk,
thatidentifiestheapplication.AcompleteapplicationmayhavemultiplecomponentsprogramswritteninCorC++,
aswellasscriptswritteninshorawk.Allofthecomponentsusethesametextdomain.
Tomakethediscussionconcrete,assumewerewritinganapplicationnamedguide.Internationalizationconsistsofthe
followingsteps,inthisorder:
267/479
29/7/2016
TheGNUAwkUsersGuide
1.Theprogrammerreviewsthesourceforallofguidescomponentsandmarkseachstringthatisacandidatefor
translation.Forexample,"`F':optionrequired"isagoodcandidatefortranslation.Atablewithstringsof
optionnamesisnot(e.g.,gawksprofileoptionshouldremainthesame,nomatterwhatthelocallanguage).
2.Theprogrammerindicatestheapplicationstextdomain("guide")tothegettextlibrary,bycallingthe
textdomain()function.
3.Messagesfromtheapplicationareextractedfromthesourcecodeandcollectedintoaportableobjecttemplate
file(guide.pot),whichliststhestringsandtheirtranslations.Thetranslationsareinitiallyempty.Theoriginal
(usuallyEnglish)messagesserveasthekeyforlookupofthetranslations.
4.Foreachlanguagewithatranslator,guide.potiscopiedtoaportableobjectfile(.po)andtranslationsarecreated
andshippedwiththeapplication.Forexample,theremightbeafr.poforaFrenchtranslation.
5.Eachlanguages.pofileisconvertedintoabinarymessageobject(.gmo)file.Amessageobjectfilecontainsthe
originalmessagesandtheirtranslationsinabinaryformatthatallowsfastlookupoftranslationsatruntime.
6.Whenguideisbuiltandinstalled,thebinarytranslationfilesareinstalledinastandardplace.
7.Fortestinganddevelopment,itispossibletotellgettexttouse.gmofilesinadifferentdirectorythanthe
standardonebyusingthebindtextdomain()function.
8.Atruntime,guidelooksupeachstringviaacalltogettext().Thereturnedstringisthetranslatedstringif
available,ortheoriginalstringifnot.
9.Ifnecessary,itispossibletoaccessmessagesfromadifferenttextdomainthantheonebelongingtothe
application,withouthavingtoswitchtheapplicationsdefaulttextdomainbackandforth.
InC(orC++),thestringmarkinganddynamictranslationlookupareaccomplishedbywrappingeachstringinacallto
gettext():
printf("%s",gettext("Don'tPanic!\n"));
Thetoolsthatextractmessagesfromsourcecodepulloutallstringsenclosedincallstogettext().
TheGNUgettextdevelopers,recognizingthattypinggettext()overandoveragainisbothpainfulanduglytolook
at,usethemacro_(anunderscore)tomakethingseasier:
/*Inthestandardheaderfile:*/
#define_(str)gettext(str)
/*Intheprogramtext:*/
printf("%s",_("Don'tPanic!\n"));
Thisreducesthetypingoverheadtojustthreeextracharactersperstringandisconsiderablyeasiertoreadaswell.
Therearelocalecategoriesfordifferenttypesoflocalerelatedinformation.Thedefinedlocalecategoriesthatgettext
knowsaboutare:
LC_MESSAGES
Textmessages.Thisisthedefaultcategoryforgettextoperations,butitispossibletosupplyadifferentone
explicitly,ifnecessary.(Itisalmostnevernecessarytosupplyadifferentcategory.)
LC_COLLATE
Textcollationinformation(i.e.,howdifferentcharactersand/orgroupsofcharacterssortinagivenlanguage).
LC_CTYPE
Charactertypeinformation(alphabetic,digit,upperorlowercase,andsoon)aswellascharacterencoding.This
informationisaccessedviathePOSIXcharacterclassesinregularexpressions,suchas/[[:alnum:]]/(see
BracketExpressions).
LC_MONETARY
Monetaryinformation,suchasthecurrencysymbol,andwhetherthesymbolgoesbeforeorafteranumber.
LC_NUMERIC
268/479
29/7/2016
TheGNUAwkUsersGuide
Numericinformation,suchaswhichcharacterstouseforthedecimalpointandthethousandsseparator.86
LC_TIME
Timeanddaterelatedinformation,suchas12or24hourclock,monthprintedbeforeorafterthedayinadate,
localmonthabbreviations,andsoon.
LC_ALL
Alloftheabove.(Nottoousefulinthecontextofgettext.)
Next:Translatori18n,Previous:Explaininggettext,Up:Internationalization[Contents][Index]
13.3InternationalizingawkPrograms
gawkprovidesthefollowingvariablesforinternationalization:
TEXTDOMAIN
Thisvariableindicatestheapplicationstextdomain.ForcompatibilitywithGNUgettext,thedefaultvalueis
"messages".
_"yourmessagehere"
Stringconstantsmarkedwithaleadingunderscorearecandidatesfortranslationatruntime.Stringconstants
withoutaleadingunderscorearenottranslated.
gawkprovidesthefollowingfunctionsforinternationalization:
dcgettext(string[,domain[,category]])
Returnthetranslationofstringintextdomaindomainforlocalecategorycategory.Thedefaultvaluefordomain
isthecurrentvalueofTEXTDOMAIN.Thedefaultvalueforcategoryis"LC_MESSAGES".
Ifyousupplyavalueforcategory,itmustbeastringequaltooneoftheknownlocalecategoriesdescribedinthe
previoussection.Youmustalsosupplyatextdomain.UseTEXTDOMAINifyouwanttousethecurrentdomain.
CAUTION:Theorderofargumentstotheawkversionofthedcgettext()functionispurposely
differentfromtheorderfortheCversion.Theawkversionsorderwaschosentobesimpleandto
allowforreasonableawkstyledefaultarguments.
dcngettext(string1,string2,number[,domain[,category]])
Returnthepluralformusedfornumberofthetranslationofstring1andstring2intextdomaindomainforlocale
categorycategory.string1istheEnglishsingularvariantofamessage,andstring2istheEnglishpluralvariantof
thesamemessage.ThedefaultvaluefordomainisthecurrentvalueofTEXTDOMAIN.Thedefaultvalueforcategory
is"LC_MESSAGES".
Thesameremarksaboutargumentorderasforthedcgettext()functionapply.
bindtextdomain(directory[,domain])
Changethedirectoryinwhichgettextlooksfor.gmofiles,incasetheywillnotorcannotbeplacedinthe
standardlocations(e.g.,duringtesting).Returnthedirectoryinwhichdomainisbound.
ThedefaultdomainisthevalueofTEXTDOMAIN.Ifdirectoryisthenullstring(""),thenbindtextdomain()returns
thecurrentbindingforthegivendomain.
Tousethesefacilitiesinyourawkprogram,followthesesteps:
269/479
29/7/2016
TheGNUAwkUsersGuide
1.SetthevariableTEXTDOMAINtothetextdomainofyourprogram.ThisisbestdoneinaBEGINrule(see
BEGIN/END),oritcanalsobedoneviathevcommandlineoption(seeOptions):
BEGIN{
TEXTDOMAIN="guide"
2.Markalltranslatablestringswithaleadingunderscore(_)character.Itmustbeadjacenttotheopeningquoteof
thestring.Forexample:
print_"hello,world"
x=_"yougoofed"
printf(_"Numberofusersis%d\n",nusers)
3.Ifyouarecreatingstringsdynamically,youcanstilltranslatethem,usingthedcgettext()builtinfunction:87
if(groggy)
message=dcgettext("%dcustomersdisturbingme\n","adminprog")
else
message=dcgettext("enjoying%dcustomers\n","adminprog")
printf(message,ncustomers)
Here,thecalltodcgettext()suppliesadifferenttextdomain("adminprog")inwhichtofindthemessage,butit
usesthedefault"LC_MESSAGES"category.
Thepreviousexampleonlyworksifncustomersisgreaterthanone.Thisexamplewouldbebetterdonewith
dcngettext():
if(groggy)
message=dcngettext("%dcustomerdisturbingme\n",
"%dcustomersdisturbingme\n","adminprog")
else
message=dcngettext("enjoying%dcustomer\n",
"enjoying%dcustomers\n","adminprog")
printf(message,ncustomers)
4.Duringdevelopment,youmightwanttoputthe.gmofileinaprivatedirectoryfortesting.Thisisdonewiththe
bindtextdomain()builtinfunction:
BEGIN{
TEXTDOMAIN="guide"#ourtextdomain
if(Testing){
#wheretofindourfiles
bindtextdomain("testdir")
#joeisinchargeofadminprog
bindtextdomain("../joe/testdir","adminprog")
}
SeeI18NExample,foranexampleprogramshowingthestepstocreateandusetranslationsfromawk.
Next:I18NExample,Previous:Programmeri18n,Up:Internationalization[Contents][Index]
13.4TranslatingawkPrograms
Onceaprogramstranslatablestringshavebeenmarked,theymustbeextractedtocreatetheinitial.potfile.Aspartof
translation,itisoftenhelpfultorearrangetheorderinwhichargumentstoprintfareoutput.
gawksgenpotcommandlineoptionextractsthemessagesandisdiscussednext.Afterthat,printfsabilityto
rearrangetheorderforprintfargumentsatruntimeiscovered.
270/479
29/7/2016
TheGNUAwkUsersGuide
StringExtraction: Extractingmarkedstrings.
PrintfOrdering: Rearrangingprintfarguments.
I18NPortability: awklevelportabilityissues.
Next:PrintfOrdering,Up:Translatori18n[Contents][Index]
13.4.1ExtractingMarkedStrings
Onceyourawkprogramisworking,andallthestringshavebeenmarkedandyouveset(andperhapsbound)thetext
domain,itistimetoproducetranslations.First,usethegenpotcommandlineoptiontocreatetheinitial.potfile:
gawkgenpotfguide.awk>guide.pot
Whenrunwithgenpot,gawkdoesnotexecuteyourprogram.Instead,itparsesitasusualandprintsallmarked
stringstostandardoutputintheformatofaGNUgettextPortableObjectfile.Alsoincludedintheoutputareany
constantstringsthatappearasthefirstargumenttodcgettext()orasthefirstandsecondargumenttodcngettext().88
Youshoulddistributethegenerated.potfilewithyourawkprogramtranslatorswilleventuallyuseittoprovideyou
translationsthatyoucanalsothendistribute.SeeI18NExample,forthefulllistofstepstogothroughtocreateandtest
translationsforguide.
Next:I18NPortability,Previous:StringExtraction,Up:Translatori18n[Contents][Index]
13.4.2RearrangingprintfArguments
Formatstringsforprintfandsprintf()(seePrintf)presentaspecialproblemfortranslation.Considerthefollowing:89
printf(_"String`%s'has%dcharacters\n",
string,length(string)))
ApossibleGermantranslationforthismightbe:
"%dZeichenlangistdieZeichenkette`%s'\n"
Theproblemshouldbeobvious:theorderoftheformatspecificationsisdifferentfromtheoriginal!Eventhough
gettext()canreturnthetranslatedstringatruntime,itcannotchangetheargumentorderinthecalltoprintf.
Tosolvethisproblem,printfformatspecifiersmayhaveanadditionaloptionalelement,whichwecallapositional
specifier.Forexample:
"%2$dZeichenlangistdieZeichenkette`%1$s'\n"
Here,thepositionalspecifierconsistsofanintegercount,whichindicateswhichargumenttouse,anda$.Countsare
onebased,andtheformatstringitselfisnotincluded.Thus,inthefollowingexample,stringisthefirstargumentand
length(string)isthesecond:
$gawk'BEGIN{
>string="Don\47tPanic"
>printf"%2$dcharacterslivein\"%1$s\"\n",
>string,length(string)
>}'
|11characterslivein"Don'tPanic"
Ifpresent,positionalspecifierscomefirstintheformatspecification,beforetheflags,thefieldwidth,and/orthe
precision.
Positionalspecifierscanbeusedwiththedynamicfieldwidthandprecisioncapability:
$gawk'BEGIN{
271/479
29/7/2016
TheGNUAwkUsersGuide
>printf("%*.*s\n",10,20,"hello")
>printf("%3$*2$.*1$s\n",20,10,"hello")
>}'
|hello
|hello
NOTE:Whenusing*withapositionalspecifier,the*comesfirst,thentheintegerposition,andthen
the$.Thisissomewhatcounterintuitive.
gawkdoesnotallowyoutomixregularformatspecifiersandthosewithpositionalspecifiersinthesamestring:
$gawk'BEGIN{printf"%d%3$s\n",1,2,"hi"}'
errorgawk:cmd.line:1:fatal:mustuse`count$'onallformatsornone
NOTE:Therearesomepathologicalcasesthatgawkmayfailtodiagnose.Insuchcases,theoutputmaynot
bewhatyouexpect.Itsstillabadideatotrymixingthem,evenifgawkdoesntdetectit.
Althoughpositionalspecifierscanbeuseddirectlyinawkprograms,theirprimarypurposeistohelpinproducing
correcttranslationsofformatstringsintolanguagesdifferentfromtheoneinwhichtheprogramisfirstwritten.
Previous:PrintfOrdering,Up:Translatori18n[Contents][Index]
13.4.3awkPortabilityIssues
gawksinternationalizationfeatureswerepurposelychosentohaveaslittleimpactaspossibleontheportabilityofawk
programsthatusethemtootherversionsofawk.Considerthisprogram:
BEGIN{
TEXTDOMAIN="guide"
if(Test_Guide)#setwithv
bindtextdomain("/test/guide/messages")
print_"don'tpanic!"
}
Aswritten,itwontworkonotherversionsofawk.However,itisactuallyalmostportable,requiringverylittlechange:
AssignmentstoTEXTDOMAINwonthaveanyeffect,becauseTEXTDOMAINisnotspecialinotherawk
implementations.
NonGNUversionsofawktreatmarkedstringsastheconcatenationofavariablenamed_withthestring
followingit.90Typically,thevariable_hasthenullstring("")asitsvalue,leavingtheoriginalstringconstantas
theresult.
Bydefiningdummyfunctionstoreplacedcgettext(),dcngettext(),andbindtextdomain(),theawkprogram
canbemadetorun,butallthemessagesareoutputintheoriginallanguage.Forexample:
functionbindtextdomain(dir,domain)
{
returndir
}
functiondcgettext(string,domain,category)
{
returnstring
}
functiondcngettext(string1,string2,number,domain,category)
{
return(number==1?string1:string2)
}
Theuseofpositionalspecificationsinprintforsprintf()isnotportable.Tosupportgettext()attheClevel,
manysystemsCversionsofsprintf()dosupportpositionalspecifiers.Butitworksonlyifenougharguments
aresuppliedinthefunctioncall.Manyversionsofawkpassprintfformatsandargumentsunchangedtothe
272/479
29/7/2016
TheGNUAwkUsersGuide
underlyingClibraryversionofsprintf(),butonlyoneformatandargumentatatime.Whathappensifa
positionalspecificationisusedisanybodysguess.However,becausethepositionalspecificationsareprimarily
foruseintranslatedformatstrings,andbecausenonGNUawksneverretrievethetranslatedstring,thisshould
notbeaprobleminpractice.
Next:GawkI18N,Previous:Translatori18n,Up:Internationalization[Contents][Index]
13.5ASimpleInternationalizationExample
Nowletslookatastepbystepexampleofhowtointernationalizeandlocalizeasimpleawkprogram,usingguide.awk
asouroriginalsource:
BEGIN{
TEXTDOMAIN="guide"
bindtextdomain(".")#fortesting
print_"Don'tPanic"
print_"TheAnswerIs",42
print"Pardonme,Zaphodwho?"
}
Rungawkgenpottocreatethe.potfile:
$gawkgenpotfguide.awk>guide.pot
Thisproduces:
#:guide.awk:4
msgid"Don'tPanic"
msgstr""
#:guide.awk:5
msgid"TheAnswerIs"
msgstr""
Thisoriginalportableobjecttemplatefileissavedandreusedforeachlanguageintowhichtheapplicationistranslated.
Themsgidistheoriginalstringandthemsgstristhetranslation.
NOTE:Stringsnotmarkedwithaleadingunderscoredonotappearintheguide.potfile.
Next,themessagesmustbetranslated.HereisatranslationtoahypotheticaldialectofEnglish,calledMellow:91
$cpguide.potguidemellow.po
Addtranslationstoguidemellow.po
Followingarethetranslations:
#:guide.awk:4
msgid"Don'tPanic"
msgstr"Heyman,relax!"
#:guide.awk:5
msgid"TheAnswerIs"
msgstr"Like,thescoopis"
Thenextstepistomakethedirectorytoholdthebinarymessageobjectfileandthentocreatetheguide.mofile.We
pretendthatourfileistobeusedintheen_US.UTF8locale,becausewehavetousealocalenameknowntotheC
gettextroutines.ThedirectorylayoutshownhereisstandardforGNUgettextonGNU/Linuxsystems.Otherversions
ofgettextmayuseadifferentlayout:
$mkdiren_US.UTF8en_US.UTF8/LC_MESSAGES
273/479
29/7/2016
TheGNUAwkUsersGuide
Themsgfmtutilitydoestheconversionfromhumanreadable.pofiletomachinereadable.mofile.Bydefault,msgfmt
createsafilenamedmessages.Thisfilemustberenamedandplacedintheproperdirectory(usingtheooption)sothat
gawkcanfindit:
$msgfmtguidemellow.pooen_US.UTF8/LC_MESSAGES/guide.mo
Finally,weruntheprogramtotestit:
$gawkfguide.awk
|Heyman,relax!
|Like,thescoopis42
|Pardonme,Zaphodwho?
Ifthethreereplacementfunctionsfordcgettext(),dcngettext(),andbindtextdomain()(seeI18NPortability)areina
filenamedlibintl.awk,thenwecanrunguide.awkunchangedasfollows:
$gawkposixfguide.awkflibintl.awk
|Don'tPanic
|TheAnswerIs42
|Pardonme,Zaphodwho?
Next:I18NSummary,Previous:I18NExample,Up:Internationalization[Contents][Index]
13.6gawkCanSpeakYourLanguage
gawkitselfhasbeeninternationalizedusingtheGNUgettextpackage.(GNUgettextisdescribedincompletedetailin
GNUgettextutilities.)Asofthiswriting,thelatestversionofGNUgettextisversion0.19.4.
Ifatranslationofgawksmessagesexists,thengawkproducesusagemessages,warnings,andfatalerrorsinthelocal
language.
Previous:GawkI18N,Up:Internationalization[Contents][Index]
13.7Summary
Internationalizationmeanswritingaprogramsuchthatitcanusemultiplelanguageswithoutrequiringsource
codechanges.Localizationmeansprovidingthedatanecessaryforaninternationalizedprogramtoworkina
particularlanguage.
gawkusesGNUgettexttoletyouinternationalizeandlocalizeawkprograms.Aprogramstextdomainidentifies
theprogramforgroupingallmessagesandotherdatatogether.
Youmarkaprogramsstringsfortranslationbyprecedingthemwithanunderscore.Oncethatisdone,thestrings
areextractedintoa.potfile.Thisfileiscopiedforeachlanguageintoa.pofile,andthe.pofilesarecompiled
into.gmofilesforuseatruntime.
Youcanusepositionalspecificationswithsprintf()andprintftorearrangetheplacementofargumentvaluesin
formattedstringsandoutput.Thisisusefulforthetranslationofformatcontrolstrings.
Theinternationalizationfeatureshavebeendesignedsothattheycanbeeasilyworkedaroundinastandardawk.
gawkitselfhasbeeninternationalizedandshipswithanumberoftranslationsforitsmessages.
Next:ArbitraryPrecisionArithmetic,Previous:Internationalization,Up:Top[Contents][Index]
Itwouldbeniceifcomputerprogramsworkedperfectlythefirsttimetheywererun,butinreallife,thisrarelyhappens
forprogramsofanycomplexity.Thus,mostprogramminglanguageshavefacilitiesavailablefordebugging
programs,andnowawkisnoexception.
274/479
29/7/2016
TheGNUAwkUsersGuide
ThegawkdebuggerispurposelymodeledaftertheGNUDebugger(GDB)commandlinedebugger.Ifyouarefamiliar
withGDB,learninghowtousegawkfordebuggingyourprogramiseasy.
Debugging:
Introductiontogawkdebugger.
SampleDebuggingSession: Sampledebuggingsession.
ListofDebuggerCommands: Maindebuggercommands.
ReadlineSupport:
Readlinesupport.
Limitations:
Limitationsandfutureplans.
DebuggingSummary:
Debuggingsummary.
Next:SampleDebuggingSession,Up:Debugger[Contents][Index]
14.1IntroductiontothegawkDebugger
Thissectionintroducesdebuggingingeneralandbeginsthediscussionofdebuggingingawk.
DebuggingConcepts: DebugginginGeneral.
DebuggingTerms: AdditionalDebuggingConcepts.
AwkDebugging:
AwkDebugging.
Next:DebuggingTerms,Up:Debugging[Contents][Index]
14.1.1DebugginginGeneral
(Ifyouhaveuseddebuggersinotherlanguages,youmaywanttoskipaheadtoAwkDebugging.)
Ofcourse,adebuggingprogramcannotremovebugsforyou,becauseithasnowayofknowingwhatyouoryourusers
considerabugversusafeature.(Sometimes,wehumanshaveahardtimewiththisourselves.)Inthatcase,what
canyouexpectfromsuchatool?Theanswertothatdependsonthelanguagebeingdebugged,butingeneral,youcan
expectatleastthefollowing:
Theabilitytowatchaprogramexecuteitsinstructionsonebyone,givingyou,theprogrammer,theopportunity
tothinkaboutwhatishappeningonatimescaleofseconds,minutes,orhours,ratherthanthenanosecondtime
scaleatwhichthecodeusuallyruns.
Theopportunitytonotonlypassivelyobservetheoperationofyourprogram,buttocontrolitandtrydifferent
pathsofexecution,withouthavingtochangeyoursourcefiles.
Thechancetoseethevaluesofdataintheprogramatanypointinexecution,andalsotochangethatdataonthe
fly,toseehowthataffectswhathappensafterward.(Thisoftenincludestheabilitytolookatinternaldata
structuresbesidesthevariablesyouactuallydefinedinyourcode.)
Theabilitytoobtainadditionalinformationaboutyourprogramsstateorevenitsinternalstructure.
Allofthesetoolsprovideagreatamountofhelpinusingyourownskillsandunderstandingofthegoalsofyour
programtofindwhereitisgoingwrong(or,forthatmatter,tobettercomprehendaperfectlyfunctionalprogramthat
youorsomeoneelsewrote).
Next:AwkDebugging,Previous:DebuggingConcepts,Up:Debugging[Contents][Index]
14.1.2DebuggingConcepts
Beforedivingintothedetails,weneedtointroduceseveralimportantconceptsthatapplytojustaboutalldebuggers.
Thefollowinglistdefinestermsusedthroughouttherestofthischapter:
Stackframe
275/479
29/7/2016
TheGNUAwkUsersGuide
Programsgenerallycallfunctionsduringthecourseoftheirexecution.Onefunctioncancallanother,ora
functioncancallitself(recursion).Youcanviewthechainofcalledfunctions(mainprogramcallsA,whichcalls
B,whichcallsC),asastackofexecutingfunctions:thecurrentlyrunningfunctionisthetopmostoneonthe
stack,andwhenitfinishes(returns),thenextonedownthenbecomestheactivefunction.Suchastackistermeda
callstack.
Foreachfunctiononthecallstack,thesystemmaintainsadataareathatcontainsthefunctionsparameters,local
variables,andreturnvalue,aswellasanyotherbookkeepinginformationneededtomanagethecallstack.This
dataareaistermedastackframe.
gawkalsofollowsthismodel,andgivesyouaccesstothecallstackandtoeachstackframe.Youcanseethecall
stack,aswellasfromwhereeachfunctiononthestackwasinvoked.Commandsthatprintthecallstackprint
informationabouteachstackframe(asdetailedlateron).
Breakpoint
Duringdebugging,youoftenwishtolettheprogramrununtilitreachesacertainpoint,andthencontinue
executionfromthereonestatement(orinstruction)atatime.Thewaytodothisistosetabreakpointwithinthe
program.Abreakpointiswheretheexecutionoftheprogramshouldbreakoff(stop),sothatyoucantakeover
controloftheprogramsexecution.Youcanaddandremoveasmanybreakpointsasyoulike.
Watchpoint
Awatchpointissimilartoabreakpoint.Thedifferenceisthatbreakpointsareorientedaroundthecode:stop
whenacertainpointinthecodeisreached.Awatchpoint,however,specifiesthatprogramexecutionshouldstop
whenadatavalueischanged.Thisisuseful,assometimesithappensthatavariablereceivesanerroneousvalue,
anditshardtotrackdownwherethishappensjustbylookingatthecode.Byusingawatchpoint,youcanstop
wheneveravariableisassignedto,andusuallyfindtheerrantcodequitequickly.
Previous:DebuggingTerms,Up:Debugging[Contents][Index]
14.1.3awkDebugging
Debugginganawkprogramhassomespecificaspectsthatarenotsharedwithprogramswritteninotherlanguages.
Firstofall,thefactthatawkprogramsusuallytakeinputlinebylinefromafileorfilesandoperateonthoselinesusing
specificrulesmakesitespeciallyusefultoorganizeviewingtheexecutionoftheprogramintermsoftheserules.Aswe
willsee,eachawkruleistreatedalmostlikeafunctioncall,withitsownspecificblockofinstructions.
Inaddition,becauseawkisbydesignaveryconciselanguage,itiseasytolosesightofeverythingthatisgoingon
insideeachlineofawkcode.Thedebuggerprovidestheopportunitytolookattheindividualprimitiveinstructions
carriedoutbythehigherlevelawkcommands.
Next:ListofDebuggerCommands,Previous:Debugging,Up:Debugger[Contents][Index]
14.2SamplegawkDebuggingSession
Inordertoillustratetheuseofgawkasadebugger,letslookatasampledebuggingsession.Wewillusetheawk
implementationofthePOSIXuniqcommanddescribedearlier(seeUniqProgram)asourexample.
DebuggerInvocation: HowtoStarttheDebugger.
FindingTheBug:
FindingtheBug.
Next:FindingTheBug,Up:SampleDebuggingSession[Contents][Index]
276/479
29/7/2016
TheGNUAwkUsersGuide
14.2.1HowtoStarttheDebugger
Startingthedebuggerisalmostexactlylikerunninggawknormally,exceptyouhavetopassanadditionaloption,
debug,orthecorrespondingshortoption,D.Thefile(s)containingtheprogramandanysupportingcodearegivenon
thecommandlineasargumentstooneormorefoptions.(gawkisnotdesignedtodebugcommandlineprograms,only
programscontainedinfiles.)Inourcase,weinvokethedebuggerlikethis:
$gawkDfgetopt.awkfjoin.awkfuniq.awk1inputfile
wherebothgetopt.awkanduniq.awkarein$AWKPATH.(ExperiencedusersofGDBorsimilardebuggersshouldnotethat
thissyntaxisslightlydifferentfromwhatyouareusedto.Withthegawkdebugger,yougivetheargumentsforrunning
theprograminthecommandlinetothedebuggerratherthanaspartoftheruncommandatthedebuggerprompt.)The
1isanoptiontouniq.awk.
Insteadofimmediatelyrunningtheprogramoninputfile,asgawkwouldordinarilydo,thedebuggermerelyloadsall
theprogramsourcefiles,compilestheminternally,andthengivesusaprompt:
gawk>
fromwhichwecanissuecommandstothedebugger.Atthispoint,nocodehasbeenexecuted.
Previous:DebuggerInvocation,Up:SampleDebuggingSession[Contents][Index]
14.2.2FindingtheBug
Letssaythatwearehavingaproblemusing(afaultyversionof)uniq.awkinthefieldskippingmode,anditdoesnt
seemtobecatchinglineswhichshouldbeidenticalwhenskippingthefirstfield,suchas:
awkisawonderfulprogram!
gawkisawonderfulprogram!
Thiscouldhappenifwewerethinking(Clike)ofthefieldsinarecordasbeingnumberedinazerobasedfashion,so
insteadofthelines:
wewrote:
clast=join(alast,fcount,n)
cline=join(aline,fcount,m)
Thefirstthingweusuallywanttodowhentryingtoinvestigateaproblemlikethisistoputabreakpointintheprogram
sothatwecanwatchitatworkandcatchwhatitisdoingwrong.Areasonablespotforabreakpointinuniq.awkisatthe
beginningofthefunctionare_equal(),whichcomparesthecurrentlinewiththepreviousone.Tosetthebreakpoint,use
theb(breakpoint)command:
gawk>bare_equal
|Breakpoint1setatfileàwklib/eg/prog/uniq.awk',line63
Thedebuggertellsusthefileandlinenumberwherethebreakpointis.Nowtyperorrunandtheprogramrunsuntil
ithitsthebreakpointforthefirsttime:
gawk>r
|Startingprogram:
|StoppinginRule...
|Breakpoint1,are_equal(n,m,clast,cline,alast,aline)
atàwklib/eg/prog/uniq.awk':63
|63if(fcount==0&&charcount==0)
gawk>
277/479
29/7/2016
TheGNUAwkUsersGuide
Nowwecanlookatwhatsgoingoninsideourprogram.Firstofall,letsseehowwegottowhereweare.Atthe
prompt,wetypebt(shortforbacktrace),andthedebuggerrespondswithalistingofthecurrentstackframes:
gawk>bt
|#0are_equal(n,m,clast,cline,alast,aline)
atàwklib/eg/prog/uniq.awk':68
|#1inmain()atàwklib/eg/prog/uniq.awk':88
Thistellsusthatare_equal()wascalledbythemainprogramatline88ofuniq.awk.(Thisisnotabigsurprise,because
thisistheonlycalltoare_equal()intheprogram,butinmorecomplexprograms,knowingwhocalledafunctionand
withwhatparameterscanbethekeytofindingthesourceoftheproblem.)
Nowthatwereinare_equal(),wecanstartlookingatthevaluesofsomevariables.Letssaywetypepn( pisshort
forprint).Wewouldexpecttoseethevalueofn,aparametertoare_equal().Actually,thedebuggergivesus:
gawk>pn
|n=untypedvariable
Inthiscase,nisanuninitializedlocalvariable,becausethefunctionwascalledwithoutarguments(seeFunctionCalls).
Amoreusefulvariabletodisplaymightbethecurrentrecord:
gawk>p$0
|$0="gawkisawonderfulprogram!"
Thismightbeabitpuzzlingatfirst,asthisisthesecondlineofourtestinput.LetslookatNR:
gawk>pNR
|NR=2
Sowecanseethatare_equal()wasonlycalledforthesecondrecordofthefile.Ofcourse,thisisbecauseourprogram
containsaruleforNR==1:
NR==1{
last=$0
next
}
OK,letsjustcheckthatthatruleworkedcorrectly:
gawk>plast
|last="awkisawonderfulprogram!"
Everythingwehavedonesofarhasverifiedthattheprogramhasworkedasplanned,uptoandincludingthecallto
are_equal(),sotheproblemmustbeinsidethisfunction.Toinvestigatefurther,wemustbeginsteppingthroughthe
linesofare_equal().Westartbytypingn(fornext):
gawk>n
|66if(fcount>0){
Thistellsusthatgawkisnowreadytoexecuteline66,whichdecideswhethertogivethelinesthespecialfield
skippingtreatmentindicatedbythe1commandlineoption.(Noticethatweskippedfromwherewewerebefore,at
line63,tohere,becausetheconditioninline63,if(fcount==0&&charcount==0),wasfalse.)
Continuingtostep,wenowgettothesplittingofthecurrentandlastrecords:
gawk>n
|67n=split(last,alast)
gawk>n
|68m=split($0,aline)
Atthispoint,weshouldbecurioustoseewhatourrecordsweresplitinto,sowetrytolook:
gawk>pnmalastaline
278/479
29/7/2016
TheGNUAwkUsersGuide
|n=5
|m=untypedvariable
|alast=array,5elements
|aline=untypedvariable
(Thepcommandcantakemorethanoneargument,similartoawksprintstatement.)
Thisiskindofdisappointing,though.Allwefoundoutisthattherearefiveelementsinalastmandalinedonthave
valuesbecauseweareatline68buthaventexecutedityet.Thisinformationisusefulenough(wenowknowthatnone
ofthewordswereaccidentallyleftout),butwhatifwewanttoseeinsidethearray?
Thefirstchoicewouldbetousesubscripts:
gawk>palast[0]
|"0"notinarrayàlast'
Oops!
gawk>palast[1]
|alast["1"]="awk"
Thiswouldbekindofslowfora100memberarray,though,sogawkprovidesashortcut(reminiscentofanother
languagenottobementioned):
gawk>p@alast
|alast["1"]="awk"
|alast["2"]="is"
|alast["3"]="a"
|alast["4"]="wonderful"
|alast["5"]="program!"
ItlookslikewegotthisfarOK.Letstakeanothersteportwo:
gawk>n
|69clast=join(alast,fcount,n)
gawk>n
|70cline=join(aline,fcount,m)
Well,hereweareatourerror(sorrytospoilthesuspense).Whatwehadinmindwastojointhefieldsstartingfromthe
secondonetomakethevirtualrecordtocompare,andifthefirstfieldwerenumberedzero,thiswouldwork.Letslook
atwhatwevegot:
gawk>pclineclast
|cline="gawkisawonderfulprogram!"
|clast="awkisawonderfulprogram!"
Hey,thoselookprettyfamiliar!Theyrejustouroriginal,unalteredinputrecords.Alittlethinking(thehumanbrainis
stillthebestdebuggingtool),andwerealizethatwewereoffbyone!
Wegetoutofthedebugger:
gawk>q
|Theprogramisrunning.Exitanyway(y/n)?y
Thenwegetintoaneditor:
andproblemsolved!
Next:ReadlineSupport,Previous:SampleDebuggingSession,Up:Debugger[Contents][Index]
279/479
29/7/2016
TheGNUAwkUsersGuide
14.3MainDebuggerCommands
Thegawkdebuggercommandsetcanbedividedintothefollowingcategories:
Breakpointcontrol
Executioncontrol
Viewingandchangingdata
Workingwiththestack
Gettinginformation
Miscellaneous
Eachofthesearediscussedinthefollowingsubsections.Inthefollowingdescriptions,commandsthatmaybe
abbreviatedshowtheabbreviationonaseconddescriptionline.Adebuggercommandnamemayalsobetruncatedif
thatpartialnameisunambiguous.Thedebuggerhasthebuiltincapabilitytoautomaticallyrepeattheprevious
commandjustbyhittingEnter.Thisworksforthecommandslist,next,nexti,step,stepi,andcontinueexecuted
withoutanyargument.
BreakpointControl:
ControlofBreakpoints.
DebuggerExecutionControl:
ControlofExecution.
ViewingAndChangingData:
ViewingandChangingData.
ExecutionStack:
DealingwiththeStack.
DebuggerInfo:
ObtainingInformationabouttheProgramandtheDebuggerState.
MiscellaneousDebuggerCommands: MiscellaneousCommands.
Next:DebuggerExecutionControl,Up:ListofDebuggerCommands[Contents][Index]
14.3.1ControlofBreakpoints
Aswesawearlier,thefirstthingyouprobablywanttodoinadebuggingsessionistogetyourbreakpointssetup,
becauseyourprogramwillotherwisejustrunasifitwasnotunderthedebugger.Thecommandsforcontrolling
breakpointsare:
break[[filename:]n|function]["expression"]
b[[filename:]n|function]["expression"]
Withoutanyargument,setabreakpointatthenextinstructiontobeexecutedintheselectedstackframe.
Argumentscanbeoneofthefollowing:
n
Setabreakpointatlinenumberninthecurrentsourcefile.
filename:n
Setabreakpointatlinenumberninsourcefilefilename.
function
Setabreakpointatentryto(thefirstinstructionof)functionfunction.
Eachbreakpointisassignedanumberthatcanbeusedtodeleteitfromthebreakpointlistusingthedelete
command.
Withabreakpoint,youmayalsosupplyacondition.Thisisanawkexpression(enclosedindoublequotes)thatthe
debuggerevaluateswheneverthebreakpointisreached.Iftheconditionistrue,thenthedebuggerstopsexecution
andpromptsforacommand.Otherwise,itcontinuesexecutingtheprogram.
280/479
29/7/2016
TheGNUAwkUsersGuide
clear[[filename:]n|function]
Withoutanyargument,deleteanybreakpointatthenextinstructiontobeexecutedintheselectedstackframe.If
theprogramstopsatabreakpoint,thisdeletesthatbreakpointsothattheprogramdoesnotstopatthatlocation
again.Argumentscanbeoneofthefollowing:
n
Deletebreakpoint(s)setatlinenumberninthecurrentsourcefile.
filename:n
Deletebreakpoint(s)setatlinenumberninsourcefilefilename.
function
Deletebreakpoint(s)setatentrytofunctionfunction.
conditionn"expression"
Addaconditiontoexistingbreakpointorwatchpointn.Theconditionisanawkexpressionenclosedindouble
quotesthatthedebuggerevaluateswheneverthebreakpointorwatchpointisreached.Iftheconditionistrue,then
thedebuggerstopsexecutionandpromptsforacommand.Otherwise,thedebuggercontinuesexecutingthe
program.Iftheconditionexpressionisnotspecified,anyexistingconditionisremoved(i.e.,thebreakpointor
watchpointismadeunconditional).
delete[n1n2][nm]
d[n1n2][nm]
Deletespecifiedbreakpointsorarangeofbreakpoints.Deletealldefinedbreakpointsifnoargumentissupplied.
disable[n1n2|nm]
Disablespecifiedbreakpointsorarangeofbreakpoints.Withoutanyargument,disableallbreakpoints.
enable[del|once][n1n2][nm]
e[del|once][n1n2][nm]
Enablespecifiedbreakpointsorarangeofbreakpoints.Withoutanyargument,enableallbreakpoints.Optionally,
youcanspecifyhowtoenablethebreakpoints:
del
Enablethebreakpointstemporarily,thendeleteeachonewhentheprogramstopsatit.
once
Enablethebreakpointstemporarily,thendisableeachonewhentheprogramstopsatit.
ignorencount
Ignorebreakpointnumbernthenextcounttimesitishit.
tbreak[[filename:]n|function]
t[[filename:]n|function]
Setatemporarybreakpoint(enabledforonlyonestop).Theargumentsarethesameasforbreak.
Next:ViewingAndChangingData,Previous:BreakpointControl,Up:ListofDebuggerCommands[Contents][Index]
281/479
29/7/2016
TheGNUAwkUsersGuide
14.3.2ControlofExecution
Nowthatyourbreakpointsareready,youcanstartrunningtheprogramandobservingitsbehavior.Therearemore
commandsforcontrollingexecutionoftheprogramthanwesawinourearlierexample:
commands[n]
silent
end
Setalistofcommandstobeexecuteduponstoppingatabreakpointorwatchpoint.nisthebreakpointor
watchpointnumber.Withoutanumber,thelastonesetisused.Theactualcommandsfollow,startingonthenext
line,andterminatedbytheendcommand.Ifthecommandsilentisinthelist,theusualmessagesaboutstopping
atabreakpointandthesourcelinearenotprinted.Anycommandinthelistthatresumesexecution(e.g.,
continue)terminatesthelist(animplicitend),andsubsequentcommandsareignored.Forexample:
gawk>commands
>silent
>printf"Asilentbreakpoint;i=%d\n",i
>infolocals
>seti=10
>continue
>end
gawk>
continue[count]
c[count]
Resumeprogramexecution.Ifcontinuedfromabreakpointandcountisspecified,ignorethebreakpointatthat
locationthenextcounttimesbeforestopping.
finish
Executeuntiltheselectedstackframereturns.Printthereturnedvalue.
next[count]
n[count]
Continueexecutiontothenextsourceline,steppingoverfunctioncalls.Theargumentcountcontrolshowmany
timestorepeattheaction,asinstep.
nexti[count]
ni[count]
Executeone(orcount)instruction(s),steppingoverfunctioncalls.
return[value]
Cancelexecutionofafunctioncall.Ifvalue(eitherastringoranumber)isspecified,itisusedasthefunctions
returnvalue.Ifusedinaframeotherthantheinnermostone(thecurrentlyexecutingfunctioni.e.,framenumber
0),discardallinnerframesinadditiontotheselectedone,andthecallerofthatframebecomestheinnermost
frame.
run
r
Start/restartexecutionoftheprogram.Whenrestarting,thedebuggerretainsthecurrentbreakpoints,watchpoints,
commandhistory,automaticdisplayvariables,anddebuggeroptions.
step[count]
s[count]
282/479
29/7/2016
TheGNUAwkUsersGuide
Continueexecutionuntilcontrolreachesadifferentsourcelineinthecurrentstackframe,steppinginsideany
functioncalledwithintheline.Iftheargumentcountissupplied,stepsthatmanytimesbeforestopping,unlessit
encountersabreakpointorwatchpoint.
stepi[count]
si[count]
Executeone(orcount)instruction(s),steppinginsidefunctioncalls.(Forillustrationofwhatismeantbyan
instructioningawk,seetheoutputshownunderdumpinMiscellaneousDebuggerCommands.)
until[[filename:]n|function]
u[[filename:]n|function]
Withoutanyargument,continueexecutionuntilalinepastthecurrentlineinthecurrentstackframeisreached.
Withanargument,continueexecutionuntilthespecifiedlocationisreached,orthecurrentstackframereturns.
Next:ExecutionStack,Previous:DebuggerExecutionControl,Up:ListofDebuggerCommands[Contents][Index]
14.3.3ViewingandChangingData
Thecommandsforviewingandchangingvariablesinsideofgawkare:
display[var|$n]
Addvariablevar(orfield$n)tothedisplaylist.Thevalueofthevariableorfieldisdisplayedeachtimethe
programstops.Eachvariableaddedtothelistisidentifiedbyauniquenumber:
gawk>displayx
|10:x=1
Thisdisplaystheassigneditemnumber,thevariablename,anditscurrentvalue.Ifthedisplayvariablereferstoa
functionparameter,itissilentlydeletedfromthelistassoonastheexecutionreachesacontextwherenosuch
variableofthegivennameexists.Withoutargument,displaydisplaysthecurrentvaluesofitemsonthelist.
eval"awkstatements"
Evaluateawkstatementsinthecontextoftherunningprogram.Youcandoanythingthatanawkprogramwould
do:assignvaluestovariables,callfunctions,andsoon.
evalparam,
awkstatements
end
Thisformofevalissimilar,butitallowsyoutodefinelocalvariablesthatexistinthecontextoftheawk
statements,insteadofusingvariablesorfunctionparametersdefinedbytheprogram.
printvar1[,var2]
pvar1[,var2]
Printthevalueofagawkvariableorfield.Fieldsmustbereferencedbyconstants:
gawk>print$3
Thisprintsthethirdfieldintheinputrecord(ifthespecifiedfielddoesnotexist,itprintsNullfield).A
variablecanbeanarrayelement,withthesubscriptsbeingconstantstringvalues.Toprintthecontentsofan
array,prefixthenameofthearraywiththe@symbol:
gawk>print@a
283/479
29/7/2016
TheGNUAwkUsersGuide
Thisprintstheindicesandthecorrespondingvaluesforallelementsinthearraya.
printfformat[,arg]
Printformattedtext.Theformatmayincludeescapesequences,suchas\n(seeEscapeSequences).Nonewline
isprintedunlessoneisspecified.
setvar=value
Assignaconstant(numberorstring)valuetoanawkvariableorfield.Stringvaluesmustbeenclosedbetween
doublequotes("").
Youcanalsosetspecialawkvariables,suchasFS,NF,NR,andsoon.
watchvar|$n["expression"]
wvar|$n["expression"]
Addvariablevar(orfield$n)tothewatchlist.Thedebuggerthenstopswheneverthevalueofthevariableor
fieldchanges.Eachwatcheditemisassignedanumberthatcanbeusedtodeleteitfromthewatchlistusingthe
unwatchcommand.
Withawatchpoint,youmayalsosupplyacondition.Thisisanawkexpression(enclosedindoublequotes)that
thedebuggerevaluateswheneverthewatchpointisreached.Iftheconditionistrue,thenthedebuggerstops
executionandpromptsforacommand.Otherwise,gawkcontinuesexecutingtheprogram.
undisplay[n]
Removeitemnumbern(orallitems,ifnoargument)fromtheautomaticdisplaylist.
unwatch[n]
Removeitemnumbern(orallitems,ifnoargument)fromthewatchlist.
Next:DebuggerInfo,Previous:ViewingAndChangingData,Up:ListofDebuggerCommands[Contents][Index]
14.3.4WorkingwiththeStack
Wheneveryourunaprogramthatcontainsanyfunctioncalls,gawkmaintainsastackofallofthefunctioncallsleading
uptowheretheprogramisrightnow.Youcanseehowyougottowhereyouare,andalsomovearoundinthestackto
seewhatthestateofthingswasinthefunctionsthatcalledtheoneyouarein.Thecommandsfordoingthisare:
backtrace[count]
bt[count]
where[count]
Printabacktraceofallfunctioncalls(stackframes),orinnermostcountframesifcount>0.Printtheoutermost
countframesifcount<0.Thebacktracedisplaysthenameandargumentstoeachfunction,thesourcefilename,
andthelinenumber.ThealiaswhereforbacktraceisprovidedforlongtimeGDBuserswhomaybeusedtothat
command.
down[count]
Movecount(default1)framesdownthestacktowardtheinnermostframe.Thenselectandprinttheframe.
frame[n]
f[n]
Selectandprintstackframen.Frame0isthecurrentlyexecuting,orinnermost,frame(functioncall)frame1is
284/479
29/7/2016
TheGNUAwkUsersGuide
theframethatcalledtheinnermostone.Thehighestnumberedframeistheoneforthemainprogram.Theprinted
informationconsistsoftheframenumber,functionandargumentnames,sourcefile,andthesourceline.
up[count]
Movecount(default1)framesupthestacktowardtheoutermostframe.Thenselectandprinttheframe.
Next:MiscellaneousDebuggerCommands,Previous:ExecutionStack,Up:ListofDebuggerCommands[Contents]
[Index]
14.3.5ObtainingInformationAbouttheProgramandtheDebuggerState
Besideslookingatthevaluesofvariables,thereisoftenaneedtogetothersortsofinformationaboutthestateofyour
programandofthedebuggingenvironmentitself.Thegawkdebuggerhasonecommandthatprovidesthisinformation,
appropriatelycalledinfo.infoisusedwithoneofanumberofargumentsthattellitexactlywhatyouwanttoknow:
infowhat
iwhat
Thevalueforwhatshouldbeoneofthefollowing:
args
Listargumentsoftheselectedframe.
break
Listallcurrentlysetbreakpoints.
display
Listallitemsintheautomaticdisplaylist.
frame
Giveadescriptionoftheselectedstackframe.
functions
Listallfunctiondefinitionsincludingsourcefilenamesandlinenumbers.
locals
Listlocalvariablesoftheselectedframe.
source
Printthenameofthecurrentsourcefile.Eachtimetheprogramstops,thecurrentsourcefileisthefile
containingthecurrentinstruction.Whenthedebuggerfirststarts,thecurrentsourcefileisthefirstfile
includedviathefoption.Thelistfilename:linenocommandcanbeusedatanytimetochangethe
currentsource.
sources
Listallprogramsources.
variables
Listallglobalvariables.
285/479
29/7/2016
TheGNUAwkUsersGuide
watch
Listallitemsinthewatchlist.
Additionalcommandsgiveyoucontroloverthedebugger,theabilitytosavethedebuggersstate,andtheabilitytorun
debuggercommandsfromafile.Thecommandsare:
option[name[=value]]
o[name[=value]]
Withoutanargument,displaytheavailabledebuggeroptionsandtheircurrentvalues.optionnameshowsthe
currentvalueofthenamedoption.optionname=valueassignsanewvaluetothenamedoption.Theavailable
optionsare:
history_size
Setthemaximumnumberoflinestokeepinthehistoryfile./.gawk_history.Thedefaultis100.
listsize
Specifythenumberoflinesthatlistprints.Thedefaultis15.
outfile
Sendgawkoutputtoafiledebuggeroutputstillgoestostandardoutput.Anemptystring("")resetsoutput
tostandardoutput.
prompt
Changethedebuggerprompt.Thedefaultisgawk>.
save_history[on|off]
Savecommandhistorytofile./.gawk_history.Thedefaultison.
save_options[on|off]
Savecurrentoptionstofile./.gawkrcuponexit.Thedefaultison.Optionsarereadbackintothenext
sessionuponstartup.
trace[on|off]
Turninstructiontracingonoroff.Thedefaultisoff.
savefilename
Savethecommandsfromthecurrentsessiontothegivenfilename,sothattheycanbereplayedusingthesource
command.
sourcefilename
Runcommand(s)fromafileanerrorinanycommanddoesnotterminateexecutionofsubsequentcommands.
Comments(linesstartingwith#)areallowedinacommandfile.Emptylinesareignoredtheydonotrepeatthe
lastcommand.Youcantrestarttheprogrambyhavingmorethanoneruncommandinthefile.Also,thelistof
commandsmayincludeadditionalsourcecommandshowever,thegawkdebuggerwillnotsourcethesamefile
morethanonceinordertoavoidinfiniterecursion.
Inadditionto,orinsteadof,thesourcecommand,youcanusetheDfileordebug=filecommandlineoptions
toexecutecommandsfromafilenoninteractively(seeOptions).
286/479
29/7/2016
TheGNUAwkUsersGuide
Previous:DebuggerInfo,Up:ListofDebuggerCommands[Contents][Index]
14.3.6MiscellaneousCommands
Thereareafewmorecommandsthatdonotfitintothepreviouscategories,asfollows:
dump[filename]
Dumpbytecodeoftheprogramtostandardoutputortothefilenamedinfilename.Thisprintsarepresentationof
theinternalinstructionsthatgawkexecutestoimplementtheawkcommandsinaprogram.Thiscanbevery
enlightening,asthefollowingpartialdumpofDavideBrinisobfuscatedcode(seeSignatureProgram)
demonstrates:
gawk>dump
|#BEGIN
|
|[1:0xfcd340]Op_rule:[in_rule=BEGIN][source_file=brini.awk]
|[1:0xfcc240]Op_push_i:"~"[MALLOC|STRING|STRCUR]
|[1:0xfcc2a0]Op_push_i:"~"[MALLOC|STRING|STRCUR]
|[1:0xfcc280]Op_match:
|[1:0xfcc1e0]Op_store_var:O
|[1:0xfcc2e0]Op_push_i:"=="[MALLOC|STRING|STRCUR]
|[1:0xfcc340]Op_push_i:"=="[MALLOC|STRING|STRCUR]
|[1:0xfcc320]Op_equal:
|[1:0xfcc200]Op_store_var:o
|[1:0xfcc380]Op_push:o
|[1:0xfcc360]Op_plus_i:0[MALLOC|NUMCUR|NUMBER]
|[1:0xfcc220]Op_push_lhs:o[do_reference=true]
|[1:0xfcc300]Op_assign_plus:
|[:0xfcc2c0]Op_pop:
|[1:0xfcc400]Op_push:O
|[1:0xfcc420]Op_push_i:""[MALLOC|STRING|STRCUR]
|[:0xfcc4a0]Op_no_op:
|[1:0xfcc480]Op_push:O
|[:0xfcc4c0]Op_concat:[expr_count=3][concat_flag=0]
|[1:0xfcc3c0]Op_store_var:x
|[1:0xfcc440]Op_push_lhs:X[do_reference=true]
|[1:0xfcc3a0]Op_postincrement:
|[1:0xfcc4e0]Op_push:x
|[1:0xfcc500]Op_plus:
|[1:0xfcc560]Op_plus:
|[1:0xfcc460]Op_leq:
|[:0xfcc5c0]Op_jmp_false:[target_jmp=0xfcc5e0]
|[1:0xfcc600]Op_push_i:"%c"[MALLOC|STRING|STRCUR]
|[:0xfcc660]Op_no_op:
|[1:0xfcc520]Op_assign_concat:c
|[:0xfcc620]Op_jmp:[target_jmp=0xfcc440]
|
|
|[2:0xfcc5a0]Op_K_printf:[expr_count=17][redir_type=""]
|[:0xfcc1c0]Op_atexit:
|[:0xfcc640]Op_stop:
|[:0xfcd150]Op_after_beginfile:
|[:0xfcc1a0]Op_after_endfile:
gawk>
help
h
Printalistofallofthegawkdebuggercommandswithashortsummaryoftheirusage.helpcommandprintsthe
informationaboutthecommandcommand.
list[|+|n|filename:n|nm|function]
l[|+|n|filename:n|nm|function]
Printthespecifiedlines(default15)fromthecurrentsourcefileorthefilenamedfilename.Thepossible
287/479
29/7/2016
TheGNUAwkUsersGuide
argumentstolistareasfollows:
(Minus)
Printlinesbeforethelineslastprinted.
+
Printlinesafterthelineslastprinted.listwithoutanyargumentdoesthesamething.
n
Printlinescenteredaroundlinenumbern.
nm
Printlinesfromntom.
filename:n
Printlinescenteredaroundlinenumberninsourcefilefilename.Thiscommandmaychangethecurrent
sourcefile.
function
Printlinescenteredaroundthebeginningofthefunctionfunction.Thiscommandmaychangethecurrent
sourcefile.
quit
q
Exitthedebugger.Debuggingisgreatfun,butsometimesweallhavetotendtootherobligationsinlife,and
sometimeswefindthebugandarefreetogoontothenextone!Aswesawearlier,ifyouarerunningaprogram,
thedebuggerwarnsyouwhenyoutypeqorquit,tomakesureyoureallywanttoquit.
trace[on|off]
Turnonoroffcontinuousprintingoftheinstructionsthatareabouttobeexecuted,alongwiththeawklinesthey
implement.Thedefaultisoff.
Itistobehopedthatmostoftheopcodesintheseinstructionsarefairlyselfexplanatory,andusingstepiand
nextiwhiletraceisonwillmakethemintofamiliarfriends.
Next:Limitations,Previous:ListofDebuggerCommands,Up:Debugger[Contents][Index]
14.4ReadlineSupport
IfgawkiscompiledwiththeGNUReadlinelibrary,youcantakeadvantageofthatlibraryscommandcompletionand
historyexpansionfeatures.Thefollowingtypesofcompletionareavailable:
Commandcompletion
Commandnames.
Sourcefilenamecompletion
Sourcefilenames.Relevantcommandsarebreak,clear,list,tbreak,anduntil.
Argumentcompletion
288/479
29/7/2016
TheGNUAwkUsersGuide
Nonnumericargumentstoacommand.Relevantcommandsareenableandinfo.
Variablenamecompletion
Globalvariablenames,andfunctionargumentsinthecurrentcontextiftheprogramisrunning.Relevant
commandsaredisplay,print,set,andwatch.
Next:DebuggingSummary,Previous:ReadlineSupport,Up:Debugger[Contents][Index]
14.5Limitations
Wehopeyoufindthegawkdebuggerusefulandenjoyabletoworkwith,butaswithanyprogram,especiallyinitsearly
releases,itstillhassomelimitations.Afewthatitsworthbeingawareofare:
Atthispoint,thedebuggerdoesnotgiveadetailedexplanationofwhatyoudidwrongwhenyoutypein
somethingitdoesntlike.Rather,itjustrespondssyntaxerror.Whenyoudofigureoutwhatyourmistakewas,
though,youllfeellikearealguru.
IfyouperusedthedumpofopcodesinMiscellaneousDebuggerCommands(orifyouarealreadyfamiliarwith
gawkinternals),youwillrealizethatmuchoftheinternalmanipulationofdataingawk,asinmanyinterpreters,is
doneonastack.Op_push,Op_pop,andthelikearethebreadandbutterofmostgawkcode.
Unfortunately,asofnow,thegawkdebuggerdoesnotallowyoutoexaminethestackscontents.Thatis,the
intermediateresultsofexpressionevaluationareonthestack,butcannotbeprinted.Rather,onlyvariablesthat
aredefinedintheprogramcanbeprinted.Ofcourse,aworkaroundforthisistousemoreexplicitvariablesatthe
debuggingstageandthenchangebacktoobscure,perhapsmoreoptimalcodelater.
Thereisnowaytolookinsidetheprocessofcompilingregularexpressionstoseeifyougotitright.Asanawk
programmer,youareexpectedtoknowthemeaningof/[^[:alnum:][:blank:]]/.
Thegawkdebuggerisdesignedtobeusedbyrunningaprogram(withallitsparameters)onthecommandline,as
describedinDebuggerInvocation.Thereisnoway(asofnow)toattachorbreakintoarunningprogram.This
seemsreasonableforalanguagethatisusedmainlyforquicklyexecuting,shortprograms.
Thegawkdebuggeronlyacceptssourcecodesuppliedwiththefoption.
Previous:Limitations,Up:Debugger[Contents][Index]
14.6Summary
Programsrarelyworkcorrectlythefirsttime.Findingbugsiscalleddebugging,andaprogramthathelpsyoufind
bugsisadebugger.gawkhasabuiltindebuggerthatworksverysimilarlytotheGNUDebugger,GDB.
Debuggersletyoustepthroughyourprogramonestatementatatime,examineandchangevariableandarray
values,anddoanumberofotherthingsthatletyouunderstandwhatyourprogramisactuallydoing(asopposed
towhatitissupposedtodo).
Likemostdebuggers,thegawkdebuggerworksintermsofstackframes,andletsyousetbothbreakpoints(stopat
apointinthecode)andwatchpoints(stopwhenadatavaluechanges).
Thedebuggercommandsetisfairlycomplete,providingcontroloverbreakpoints,execution,viewingand
changingdata,workingwiththestack,gettinginformation,andothertasks.
IftheGNUReadlinelibraryisavailablewhengawkiscompiled,itisusedbythedebuggertoprovidecommand
linehistoryandediting.
Next:DynamicExtensions,Previous:Debugger,Up:Top[Contents][Index]
Thischapterintroducessomebasicconceptsrelatingtohowcomputersdoarithmeticanddefinessomeimportantterms.
289/479
29/7/2016
TheGNUAwkUsersGuide
Itthenproceedstodescribefloatingpointarithmetic,whichiswhatawkusesforallitscomputations,includinga
discussionofarbitraryprecisionfloatingpointarithmetic,whichisafeatureavailableonlyingawk.Itcontinuesonto
presentarbitraryprecisionintegers,andconcludeswithadescriptionofsomepointswheregawkandthePOSIX
standardarenotquiteinagreement.
NOTE:Mostusersofgawkcansafelyskipthischapter.Butifyouwanttodoscientificcalculationswith
gawk,thisistheplacetobe.
ComputerArithmetic:
Aquickintrotocomputermath.
MathDefinitions:
Definingtermsused.
MPFRfeatures:
TheMPFRfeaturesingawk.
FPMathCaution:
Thingstoknow.
ArbitraryPrecisionIntegers:
ArbitraryPrecisionIntegerArithmeticwithgawk.
POSIXFloatingPointProblems: StandardsVersusExistingPractice.
Floatingpointsummary:
Summaryoffloatingpointdiscussion.
Next:MathDefinitions,Up:ArbitraryPrecisionArithmetic[Contents][Index]
15.1AGeneralDescriptionofComputerArithmetic
Untilnow,wehaveworkedwithdataaseithernumbersorstrings.Ultimately,however,computersrepresenteverything
intermsofbinarydigits,orbits.Adecimaldigitcantakeonanyof10values:zerothroughnine.Abinarydigitcan
takeonanyoftwovalues,zeroorone.Usingbinary,computers(andcomputersoftware)canrepresentandmanipulate
numericalandcharacterdata.Ingeneral,themorebitsyoucanusetorepresentaparticularthing,thegreatertherange
ofpossiblevaluesitcantakeon.
Moderncomputerssupportatleasttwo,andoftenmore,waystodoarithmetic.Eachkindofarithmeticusesadifferent
representation(organizationofthebits)forthenumbers.Thekindsofarithmeticthatinterestusare:
Decimalarithmetic
Thisisthekindofarithmeticyoulearnedinelementaryschool,usingpaperandpencil(and/oracalculator).In
theory,numberscanhaveanarbitrarynumberofdigitsoneitherside(orbothsides)ofthedecimalpoint,andthe
resultsofacomputationarealwaysexact.
Somemodernsystemscandodecimalarithmeticinhardware,butusuallyyouneedaspecialsoftwarelibraryto
provideaccesstotheseinstructions.Therearealsolibrariesthatdodecimalarithmeticentirelyinsoftware.
Despitethefactthatsomeusersexpectgawktobeperformingdecimalarithmetic,92itdoesnotdoso.
Integerarithmetic
Inschool,integervalueswerereferredtoaswholenumbersthatis,numberswithoutanyfractionalpart,such
as1,42,or17.Theadvantagetointegernumbersisthattheyrepresentvaluesexactly.Thedisadvantageisthat
theirrangeislimited.
Incomputers,integervaluescomeintwoflavors:signedandunsigned.Signedvaluesmaybenegativeor
positive,whereasunsignedvaluesarealwaysgreaterthanorequaltozero.
Incomputersystems,integerarithmeticisexact,butthepossiblerangeofvaluesislimited.Integerarithmeticis
generallyfasterthanfloatingpointarithmetic.
Floatingpointarithmetic
Floatingpointnumbersrepresentwhatwerecalledinschoolrealnumbers(i.e.,thosethathaveafractionalpart,
suchas3.1415927).Theadvantagetofloatingpointnumbersisthattheycanrepresentamuchlargerrangeof
valuesthancanintegers.Thedisadvantageisthattherearenumbersthattheycannotrepresentexactly.
290/479
29/7/2016
TheGNUAwkUsersGuide
Modernsystemssupportfloatingpointarithmeticinhardware,withalimitedrangeofvalues.Therearesoftware
librariesthatallowtheuseofarbitraryprecisionfloatingpointcalculations.
POSIXawkusesdoubleprecisionfloatingpointnumbers,whichcanholdmoredigitsthansingleprecision
floatingpointnumbers.gawkhasfacilitiesforperformingarbitraryprecisionfloatingpointarithmetic,whichwe
describeinmoredetailshortly.
Computersworkwithintegerandfloatingpointvaluesofdifferentranges.Integervaluesareusuallyeither32or64bits
insize.Singleprecisionfloatingpointvaluesoccupy32bits,whereasdoubleprecisionfloatingpointvaluesoccupy64
bits.Floatingpointvaluesarealwayssigned.ThepossiblerangesofvaluesareshowninTable15.1.
Numericrepresentation
32bitsignedinteger
32bitunsignedinteger
64bitsignedinteger
64bitunsignedinteger
Singleprecisionfloatingpoint
(approximate)
Doubleprecisionfloatingpoint
(approximate)
Minimumvalue
2,147,483,648
0
9,223,372,036,854,775,808
0
Maximumvalue
2,147,483,647
4,294,967,295
9,223,372,036,854,775,807
18,446,744,073,709,551,615
1.175494e38
3.402823e38
2.225074e308
1.797693e308
Table15.1:Valuerangesfordifferentnumericrepresentations
Next:MPFRfeatures,Previous:ComputerArithmetic,Up:ArbitraryPrecisionArithmetic[Contents][Index]
15.2OtherStufftoKnow
Therestofthischapterusesanumberofterms.Herearesomeinformaldefinitionsthatshouldhelpyouworkyourway
throughthematerialhere:
Accuracy
Afloatingpointcalculationsaccuracyishowcloseitcomestothereal(paperandpencil)value.
Error
Thedifferencebetweenwhattheresultofacomputationshouldbeandwhatitactuallyis.Itisbesttominimize
errorasmuchaspossible.
Exponent
Theorderofmagnitudeofavaluesomenumberofbitsinafloatingpointvaluestoretheexponent.
Inf
Aspecialvaluerepresentinginfinity.Operationsinvolvinganothernumberandinfinityproduceinfinity.
NaN
Notanumber.93Aspecialvaluethatresultsfromattemptingacalculationthathasnoanswerasarealnumber.
Insuchacase,programscaneitherreceiveafloatingpointexception,orgetNaNbackastheresult.TheIEEE754
standardrecommendsthatsystemsreturnNaN.Someexamples:
sqrt(1)
Thismakessenseintherangeofcomplexnumbers,butnotintherangeofrealnumbers,sotheresultis
291/479
29/7/2016
TheGNUAwkUsersGuide
NaN.
log(8)
8isoutofthedomainoflog(),sotheresultisNaN.
Normalized
Howthesignificand(seelaterinthislist)isusuallystored.Thevalueisadjustedsothatthefirstbitisone,and
thenthatleadingoneisassumedinsteadofphysicallystored.Thisprovidesoneextrabitofprecision.
Precision
Thenumberofbitsusedtorepresentafloatingpointnumber.Themorebits,themoredigitsyoucanrepresent.
Binaryanddecimalprecisionsarerelatedapproximately,accordingtotheformula:
prec=3.322*dps
Here,precdenotesthebinaryprecision(measuredinbits)anddps(shortfordecimalplaces)isthedecimaldigits.
Roundingmode
Hownumbersareroundedupordownwhennecessary.Moredetailsareprovidedlater.
Significand
Afloatingpointvalueconsistsofthesignificandmultipliedby10tothepoweroftheexponent.Forexample,in
1.2345e67,thesignificandis1.2345.
Stability
FromtheWikipediaarticleonnumericalstability:Calculationsthatcanbeprovennottomagnifyapproximation
errorsarecallednumericallystable.
SeetheWikipediaarticleonaccuracyandprecisionformoreinformationonsomeofthoseterms.
Onmodernsystems,floatingpointhardwareusestherepresentationandoperationsdefinedbytheIEEE754standard.
ThreeofthestandardIEEE754typesare32bitsingleprecision,64bitdoubleprecision,and128bitquadruple
precision.Thestandardalsospecifiesextendedprecisionformatstoallowgreaterprecisionsandlargerexponentranges.
(awkusesonlythe64bitdoubleprecisionformat.)
Table15.2liststheprecisionandexponentfieldvaluesforthebasicIEEE754binaryformats.
Name
Single
Double
Quadruple
Totalbits
32
64
128
Precision
24
53
113
Minimumexponent
126
1022
16382
Maximumexponent
+127
+1023
+16383
Table15.2:BasicIEEEformatvalues
NOTE:Theprecisionnumbersincludetheimpliedleadingonethatgivesthemoneextrabitofsignificand.
Next:FPMathCaution,Previous:MathDefinitions,Up:ArbitraryPrecisionArithmetic[Contents][Index]
15.3ArbitraryPrecisionArithmeticFeaturesingawk
Bydefault,gawkusesthedoubleprecisionfloatingpointvaluessuppliedbythehardwareofthesystemitrunson.
However,ifitwascompiledtodoso,gawkusestheGNUMPFRandGNUMP(GMP)librariesforarbitraryprecision
292/479
29/7/2016
TheGNUAwkUsersGuide
arithmeticonnumbers.YoucanseeifMPFRsupportisavailablelikeso:
$gawkversion
|GNUAwk4.1.2,API:1.1(GNUMPFR3.1.0p3,GNUMP5.0.2)
|Copyright(C)1989,19912015FreeSoftwareFoundation.
(Youmayseedifferentversionnumbersthanwhatsshownhere.ThatsOKwhatsimportantistoseethatGNUMPFR
andGNUMParelistedintheoutput.)
Additionally,thereareafewelementsavailableinthePROCINFOarraytoprovideinformationabouttheMPFRandGMP
libraries(seeAutoset).
TheMPFRlibraryprovidesprecisecontroloverprecisionsandroundingmodes,andgivescorrectlyrounded,
reproducible,platformindependentresults.WiththeMcommandlineoption,allfloatingpointarithmeticoperatorsand
numericfunctionscanyieldresultstoanydesiredprecisionlevelsupportedbyMPFR.
Twopredefinedvariables,PRECandROUNDMODE,providecontrolovertheworkingprecisionandtheroundingmode.The
precisionandtheroundingmodearesetgloballyforeveryoperationtofollow.SeeSettingprecision,andSettingthe
roundingmode,formoreinformation.
Next:ArbitraryPrecisionIntegers,Previous:MPFRfeatures,Up:ArbitraryPrecisionArithmetic[Contents][Index]
15.4FloatingPointArithmetic:CaveatEmptor!
Mathclassistough!
TeenTalkBarbie,July1992
Thissectionprovidesahighleveloverviewoftheissuesinvolvedwhendoinglotsoffloatingpointarithmetic.94The
discussionappliestobothhardwareandarbitraryprecisionfloatingpointarithmetic.
CAUTION:Thematerialhereispurposelygeneral.Ifyouneedtodoseriouscomputerarithmetic,you
shoulddosomeresearchfirst,andnotrelyjustonwhatwetellyou.
Inexactnessofcomputations: Floatingpointmathisnotexact.
GettingAccuracy:
Gettingmoreaccuracytakessomework.
TryToRound:
Adddigitsandround.
Settingprecision:
Howtosettheprecision.
Settingtheroundingmode: Howtosettheroundingmode.
Next:GettingAccuracy,Up:FPMathCaution[Contents][Index]
15.4.1FloatingPointArithmeticIsNotExact
Binaryfloatingpointrepresentationsandarithmeticareinexact.Simplevalueslike0.1cannotbepreciselyrepresented
usingbinaryfloatingpointnumbers,andthelimitedprecisionoffloatingpointnumbersmeansthatslightchangesin
theorderofoperationsortheprecisionofintermediatestoragecanchangetheresult.Tomakemattersworse,with
arbitraryprecisionfloatingpointarithmetic,youcansettheprecisionbeforestartingacomputation,butthenyou
cannotbesureofthenumberofsignificantdecimalplacesinthefinalresult.
Inexactrepresentation: Numbersarenotexactlyrepresented.
ComparingFPValues: Howtocomparefloatingpointvalues.
Errorsaccumulate:
Errorsgetbiggerastheygo.
Next:ComparingFPValues,Up:Inexactnessofcomputations[Contents][Index]
293/479
29/7/2016
TheGNUAwkUsersGuide
15.4.1.1ManyNumbersCannotBeRepresentedExactly
So,beforeyoustarttowriteanycode,youshouldthinkaboutwhatyoureallywantandwhatsreallyhappening.
Considerthetwonumbersinthefollowingexample:
x=0.875#1/2+1/4+1/8
y=0.425
Unlikethenumberiny,thenumberstoredinxisexactlyrepresentableinbinarybecauseitcanbewrittenasafinite
sumofoneormorefractionswhosedenominatorsareallpowersoftwo.Whengawkreadsafloatingpointnumberfrom
programsource,itautomaticallyroundsthatnumbertowhateverprecisionyourmachinesupports.Ifyoutrytoprintthe
numericcontentofavariableusinganoutputformatstringof"%.17g",itmaynotproducethesamenumberasyou
assignedtoit:
$gawk'BEGIN{x=0.875;y=0.425
>printf("%0.17g,%0.17g\n",x,y)}'
|0.875,0.42499999999999999
Oftentheerrorissosmallyoudonotevennoticeit,andifyoudo,youcanalwaysspecifyhowmuchprecisionyou
wouldlikeinyouroutput.Usuallythisisaformatstringlike"%.15g",which,whenusedinthepreviousexample,
producesanoutputidenticaltotheinput.
Next:Errorsaccumulate,Previous:Inexactrepresentation,Up:Inexactnessofcomputations[Contents][Index]
15.4.1.2BeCarefulComparingValues
Becausetheunderlyingrepresentationcanbealittlebitofffromtheexactvalue,comparingfloatingpointvaluestosee
iftheyareexactlyequalisgenerallyabadidea.Hereisanexamplewhereitdoesnotworklikeyouwouldexpect:
$gawk'BEGIN{print(0.1+12.2==12.3)}'
|0
Thegeneralwisdomwhencomparingfloatingpointvaluesistoseeiftheyarewithinsomesmallrangeofeachother
(calledadelta,ortolerance).Youhavetodecidehowsmalladeltaisimportanttoyou.Codetodothislookssomething
likethefollowing:
delta=0.00001#forexample
difference=abs(a)abs(b)#subtractthetwovalues
if(difference<delta)
#allok
else
#notok
(Weassumethatyouhaveasimpleabsolutevaluefunctionnamedabs()definedelsewhereinyourprogram.)
Previous:ComparingFPValues,Up:Inexactnessofcomputations[Contents][Index]
15.4.1.3ErrorsAccumulate
Thelossofaccuracyduringasinglecomputationwithfloatingpointnumbersusuallyisntenoughtoworryabout.
However,ifyoucomputeavaluethatistheresultofasequenceoffloatingpointoperations,theerrorcanaccumulate
andgreatlyaffectthecomputationitself.Hereisanattempttocomputethevalueofpiusingoneofitsmanyseries
representations:
BEGIN{
x=1.0/sqrt(3.0)
n=6
for(i=1;i<30;i++){
n=n*2.0
294/479
29/7/2016
TheGNUAwkUsersGuide
x=(sqrt(x*x+1)1)/x
printf("%.15f\n",n*x)
}
}
Whenrun,theearlyerrorspropagatethroughlatercomputations,causingthelooptoterminateprematurelyafter
attemptingtodividebyzero:
$gawkfpi.awk
|3.215390309173475
|3.159659942097510
|3.146086215131467
|3.142714599645573
|3.224515243534819
|2.791117213058638
|0.000000000000000
errorgawk:pi.awk:6:fatal:divisionbyzeroattempted
Hereisanadditionalexamplewheretheinaccuraciesininternalrepresentationsyieldanunexpectedresult:
$gawk'BEGIN{
>for(d=1.1;d<=1.5;d+=0.1)#loopfivetimes(?)
>i++
>printi
>}'
|4
Next:TryToRound,Previous:Inexactnessofcomputations,Up:FPMathCaution[Contents][Index]
15.4.2GettingtheAccuracyYouNeed
Canarbitraryprecisionarithmeticgiveexactresults?Therearenoeasyanswers.Thestandardrulesofalgebraoftendo
notapplywhenusingfloatingpointarithmetic.Amongotherthings,thedistributiveandassociativelawsdonothold
completely,andorderofoperationmaybeimportantforyourcomputation.Roundingerror,cumulativeprecisionloss,
andunderflowareoftentroublesome.
Whengawkteststheexpressions0.1+12.2and12.3forequalityusingthemachinedoubleprecisionarithmetic,it
decidesthattheyarenotequal!(SeeComparingFPValues.)Youcangettheresultyouwantbyincreasingtheprecision
56bitsinthiscasedoesthejob:
$gawkMvPREC=56'BEGIN{print(0.1+12.2==12.3)}'
|1
Ifaddingmorebitsisgood,perhapsaddingevenmorebitsofprecisionisbetter?Hereiswhathappensifweusean
evenlargervalueofPREC:
$gawkMvPREC=201'BEGIN{print(0.1+12.2==12.3)}'
|0
ThisisnotabugingawkorintheMPFRlibrary.Itiseasytoforgetthatthefinitenumberofbitsusedtostorethevalue
isoftenjustanapproximationafterproperrounding.Thetestforequalitysucceedsifandonlyifallbitsinthetwo
operandsareexactlythesame.Becausethisisnotnecessarilytrueafterfloatingpointcomputationswithaparticular
precisionandeffectiveroundingmode,astraighttestforequalitymaynotwork.Instead,comparethetwonumbersto
seeiftheyarewithinthedesirabledeltaofeachother.
Inapplicationswhere15orfewerdecimalplacessuffice,hardwaredoubleprecisionarithmeticcanbeadequate,andis
usuallymuchfaster.Butyouneedtokeepinmindthateveryfloatingpointoperationcansufferanewroundingerror
withcatastrophicconsequences,asillustratedbyourearlierattempttocomputethevalueofpi.Extraprecisioncan
greatlyenhancethestabilityandtheaccuracyofyourcomputationinsuchcases.
Additionally,youshouldunderstandthatrepeatedadditionisnotnecessarilyequivalenttomultiplicationinfloating
295/479
29/7/2016
TheGNUAwkUsersGuide
pointarithmetic.IntheexampleinErrorsaccumulate:
$gawk'BEGIN{
>for(d=1.1;d<=1.5;d+=0.1)#loopfivetimes(?)
>i++
>printi
>}'
|4
youmayormaynotsucceedingettingthecorrectresultbychoosinganarbitrarilylargevalueforPREC.Reformulation
oftheproblemathandisoftenthecorrectapproachinsuchsituations.
Next:Settingprecision,Previous:GettingAccuracy,Up:FPMathCaution[Contents][Index]
15.4.3TryaFewExtraBitsofPrecisionandRounding
Insteadofarbitraryprecisionfloatingpointarithmetic,oftenallyouneedisanadjustmentofyourlogicoradifferent
orderfortheoperationsinyourcalculation.Thestabilityandtheaccuracyofthecomputationofpiintheearlier
examplecanbeenhancedbyusingthefollowingsimplealgebraictransformation:
(sqrt(x*x+1)1)/xx/(sqrt(x*x+1)+1)
Aftermakingthischange,theprogramconvergestopiinunder30iterations:
$gawkfpi2.awk
|3.215390309173473
|3.159659942097501
|3.146086215131436
|3.142714599645370
|3.141873049979825
|3.141592653589797
|3.141592653589797
Next:Settingtheroundingmode,Previous:TryToRound,Up:FPMathCaution[Contents][Index]
15.4.4SettingthePrecision
gawkusesaglobalworkingprecisionitdoesnotkeeptrackoftheprecisionoraccuracyofindividualnumbers.
Performinganarithmeticoperationorcallingabuiltinfunctionroundstheresulttothecurrentworkingprecision.The
defaultworkingprecisionis53bits,whichyoucanmodifyusingthepredefinedvariablePREC.Youcanalsosetthe
valuetooneofthepredefinedcaseinsensitivestringsshowninTable15.3,toemulateanIEEE754binaryformat.
IEEE754binaryformat
"half" 16bithalfprecision
"single" Basic32bitsingleprecision
"double" Basic64bitdoubleprecision
"quad" Basic128bitquadrupleprecision
"oct"
256bitoctupleprecision
PREC
Table15.3:PredefinedprecisionstringsforPREC
Thefollowingexampleillustratestheeffectsofchangingprecisiononarithmeticoperations:
$gawkMvPREC=100'BEGIN{x=1.0e400;printx+0
>PREC="double";printx+0}'
|1e400
|0
296/479
29/7/2016
TheGNUAwkUsersGuide
CAUTION:Bewaryoffloatingpointconstants!Whenreadingafloatingpointconstantfromprogram
sourcecode,gawkusesthedefaultprecision(thatofaCdouble),unlessoverriddenbyanassignmenttothe
specialvariablePREConthecommandline,tostoreitinternallyasanMPFRnumber.Changingthe
precisionusingPRECintheprogramtextdoesnotchangetheprecisionofaconstant.
Ifyouneedtorepresentafloatingpointconstantatahigherprecisionthanthedefaultandcannotusea
commandlineassignmenttoPREC,youshouldeitherspecifytheconstantasastring,orasarational
number,wheneverpossible.Thefollowingexampleillustratesthedifferencesamongvariouswaystoprint
afloatingpointconstant:
$gawkM'BEGIN{PREC=113;printf("%0.25f\n",0.1)}'
|0.1000000000000000055511151
$gawkMvPREC=113'BEGIN{printf("%0.25f\n",0.1)}'
|0.1000000000000000000000000
$gawkM'BEGIN{PREC=113;printf("%0.25f\n","0.1")}'
|0.1000000000000000000000000
$gawkM'BEGIN{PREC=113;printf("%0.25f\n",1/10)}'
|0.1000000000000000000000000
Previous:Settingprecision,Up:FPMathCaution[Contents][Index]
15.4.5SettingtheRoundingMode
TheROUNDMODEvariableprovidesprogramlevelcontrolovertheroundingmode.ThecorrespondencebetweenROUNDMODE
andtheIEEEroundingmodesisshowninTable15.4.
Roundingmode
Roundtonearest,tiestoeven
Roundtowardpositiveinfinity
Roundtowardnegativeinfinity
Roundtowardzero
Roundtonearest,tiesawayfromzero
IEEEname
ROUNDMODE
roundTiesToEven
"N"or"n"
roundTowardPositive
"U"or"u"
roundTowardNegative
"D"or"d"
roundTowardZero
"Z"or"z"
roundTiesToAway
"A"or"a"
Table15.4:gawkroundingmodes
ROUNDMODEhasthedefaultvalue"N",whichselectstheIEEE754roundingmoderoundTiesToEven.InTable15.4,the
value"A"selectsroundTiesToAway.ThisisonlyavailableifyourversionoftheMPFRlibrarysupportsitotherwise,
settingROUNDMODEto"A"hasnoeffect.
ThedefaultmoderoundTiesToEvenisthemostpreferred,buttheleastintuitive.Thismethoddoestheobviousthingfor
mostvalues,byroundingthemupordowntothenearestdigit.Forexample,rounding1.132totwodigitsyields1.13,
androunding1.157yields1.16.
However,whenitcomestoroundingavaluethatisexactlyhalfwaybetween,thingsdonotworkthewayyouprobably
learnedinschool.Inthiscase,thenumberisroundedtothenearestevendigit.Sorounding0.125totwodigitsrounds
downto0.12,butrounding0.6875tothreedigitsroundsupto0.688.Youprobablyhavealreadyencounteredthis
roundingmodewhenusingprintftoformatfloatingpointnumbers.Forexample:
BEGIN{
x=4.5
for(i=1;i<10;i++){
x+=1.0
printf("%4.1f=>%2.0f\n",x,x)
}
}
producesthefollowingoutputwhenrunontheauthorssystem:95
297/479
29/7/2016
TheGNUAwkUsersGuide
3.5=>4
2.5=>2
1.5=>2
0.5=>0
0.5=>0
1.5=>2
2.5=>2
3.5=>4
4.5=>4
ThetheorybehindroundTiesToEvenisthatitmoreorlessevenlydistributesupwardanddownwardroundsofexact
halves,whichmightcauseanyaccumulatingroundofferrortocancelitselfout.Thisisthedefaultroundingmodefor
IEEE754computingfunctionsandoperators.
Theotherroundingmodesarerarelyused.Roundingtowardpositiveinfinity(roundTowardPositive)andtoward
negativeinfinity(roundTowardNegative)areoftenusedtoimplementintervalarithmetic,whereyouadjusttherounding
modetocalculateupperandlowerboundsfortherangeofoutput.TheroundTowardZeromodecanbeusedfor
convertingfloatingpointnumberstointegers.TheroundingmoderoundTiesToAwayroundstheresulttothenearest
numberandselectsthenumberwiththelargermagnitudeifatieoccurs.
Somenumericalanalystswilltellyouthatyourchoiceofroundingstylehastremendousimpactonthefinaloutcome,
andadviseyoutowaituntilfinaloutputforanyrounding.Instead,youcanoftenavoidroundofferrorproblemsby
settingtheprecisioninitiallytosomevaluesufficientlylargerthanthefinaldesiredprecision,sothattheaccumulation
ofroundofferrordoesnotinfluencetheoutcome.Ifyoususpectthatresultsfromyourcomputationaresensitiveto
accumulationofroundofferror,lookforasignificantdifferenceinoutputwhenyouchangetheroundingmodetobe
sure.
Next:POSIXFloatingPointProblems,Previous:FPMathCaution,Up:ArbitraryPrecisionArithmetic[Contents]
[Index]
15.5ArbitraryPrecisionIntegerArithmeticwithgawk
WhengiventheMoption,gawkperformsallintegerarithmeticusingGMParbitraryprecisionintegers.Anynumberthat
lookslikeanintegerinasourceordatafileisstoredasanarbitraryprecisioninteger.Thesizeoftheintegerislimited
onlybytheavailablememory.Forexample,thefollowingcomputes5^4^3^2,theresultofwhichisbeyondthelimitsof
ordinaryhardwaredoubleprecisionfloatingpointvalues:
$gawkM'BEGIN{
>x=5^4^3^2
>print"numberofdigits=",length(x)
>printsubstr(x,1,20),"...",substr(x,length(x)19,20)
>}'
|numberofdigits=183231
|62060698786608744707...92256259918212890625
Ifinsteadyouweretocomputethesamevalueusingarbitraryprecisionfloatingpointvalues,theprecisionneededfor
correctoutput(usingtheformulaprec=3.322*dps)wouldbe3.322x183231,or608693.
Theresultfromanarithmeticoperationwithanintegerandafloatingpointvalueisafloatingpointvaluewitha
precisionequaltotheworkingprecision.ThefollowingprogramcalculatestheeighthterminSylvesterssequence96
usingarecurrence:
$gawkM'BEGIN{
>s=2.0
>for(i=1;i<=7;i++)
>s=s*(s1)+1
>prints
>}'
|113423713055421845118910464
Theoutputdiffersfromtheactualnumber,113,423,713,055,421,844,361,000,443,becausethedefaultprecisionof53
298/479
29/7/2016
TheGNUAwkUsersGuide
bitsisnotenoughtorepresentthefloatingpointresultsexactly.Youcaneitherincreasetheprecision(100bitsis
enoughinthiscase),orreplacethefloatingpointconstant2.0withaninteger,toperformallcomputationsusing
integerarithmetictogetthecorrectoutput.
Sometimesgawkmustimplicitlyconvertanarbitraryprecisionintegerintoanarbitraryprecisionfloatingpointvalue.
ThisisprimarilybecausetheMPFRlibrarydoesnotalwaysprovidetherelevantinterfacetoprocessarbitraryprecision
integersormixedmodenumbersasneededbyanoperationorfunction.Insuchacase,theprecisionissettothe
minimumvaluenecessaryforexactconversion,andtheworkingprecisionisnotusedforthispurpose.Ifthisisnot
whatyouneedorwant,youcanemployasubterfugeandconverttheintegertofloatingpointfirst,likethis:
gawkM'BEGIN{n=13;print(n+0.0)%2.0}'
Youcanavoidthisissuealtogetherbyspecifyingthenumberasafloatingpointvaluetobeginwith:
gawkM'BEGIN{n=13.0;printn%2.0}'
Notethatforthisparticularexample,itislikelybesttojustusethefollowing:
gawkM'BEGIN{n=13;printn%2}'
Next:Floatingpointsummary,Previous:ArbitraryPrecisionIntegers,Up:ArbitraryPrecisionArithmetic[Contents]
[Index]
15.6StandardsVersusExistingPractice
Historically,awkhasconvertedanynonnumericlookingstringtothenumericvaluezero,whenrequired.Furthermore,
theoriginaldefinitionofthelanguageandtheoriginalPOSIXstandardsspecifiedthatawkonlyunderstandsdecimal
numbers(base10),andnotoctal(base8)orhexadecimalnumbers(base16).
Changesinthelanguageofthe2001and2004POSIXstandardscanbeinterpretedtoimplythatawkshouldsupport
additionalfeatures.Thesefeaturesare:
Interpretationoffloatingpointdatavaluesspecifiedinhexadecimalnotation(e.g.,0xDEADBEEF).(Note:data
values,notsourcecodeconstants.)
SupportforthespecialIEEE754floatingpointvaluesnotanumber(NaN),positiveinfinity(inf),and
negativeinfinity(inf).Inparticular,theformatforthesevaluesisasspecifiedbytheISO1999Cstandard,
whichignorescaseandcanallowimplementationdependentadditionalcharactersafterthenanandalloweither
inforinfinity.
Thefirstproblemisthatbothoftheseareclearchangestohistoricalpractice:
Thegawkmaintainerfeelsthatsupportinghexadecimalfloatingpointvalues,inparticular,isugly,andwasnever
intendedbytheoriginaldesignerstobepartofthelanguage.
Allowingcompletelyalphabeticstringstohavevalidnumericvaluesisalsoaveryseveredeparturefrom
historicalpractice.
Thesecondproblemisthatthegawkmaintainerfeelsthatthisinterpretationofthestandard,whichrequiredacertain
amountoflanguagelawyeringtoarriveatinthefirstplace,wasnotevenintendedbythestandarddevelopers.In
otherwords,Weseehowyougotwhereyouare,butwedontthinkthatthatswhereyouwanttobe.
Recognizingtheseissues,butattemptingtoprovidecompatibilitywiththeearlierversionsofthestandard,the2008
POSIXstandardaddedexplicitwordingtoallow,butnotrequire,thatawksupporthexadecimalfloatingpointvaluesand
specialvaluesfornotanumberandinfinity.
Althoughthegawkmaintainercontinuestofeelthatprovidingthosefeaturesisinadvisable,nevertheless,onsystemsthat
supportIEEEfloatingpoint,itseemsreasonabletoprovidesomewaytosupportNaNandinfinityvalues.Thesolution
implementedingawkisasfollows:
299/479
29/7/2016
TheGNUAwkUsersGuide
Withtheposixcommandlineoption,gawkbecomeshandsoff.Stringvaluesarepasseddirectlytothesystem
librarysstrtod()function,andifitsuccessfullyreturnsanumericvalue,thatiswhatsused.97Bydefinition,the
resultsarenotportableacrossdifferentsystems.Theyarealsoalittlesurprising:
$echonanny|gawkposix'{print$1+0}'
|nan
$echo0xDeadBeef|gawkposix'{print$1+0}'
|3735928559
Withoutposix,gawkinterpretsthefourstringvalues+inf,inf,+nan,andnanspecially,producingthe
correspondingspecialnumericvalues.Theleadingsignactsasignaltogawk(andtheuser)thatthevalueisreally
numeric.Hexadecimalfloatingpointisnotsupported(unlessyoualsousenondecimaldata,whichisnot
recommended).Forexample:
$echonanny|gawk'{print$1+0}'
|0
$echo+nan|gawk'{print$1+0}'
|nan
$echo0xDeadBeef|gawk'{print$1+0}'
|0
gawkignorescaseinthefourspecialvalues.Thus,+nanand+NaNarethesame.
Previous:POSIXFloatingPointProblems,Up:ArbitraryPrecisionArithmetic[Contents][Index]
15.7Summary
Mostcomputerarithmeticisdoneusingeitherintegersorfloatingpointvalues.Standardawkusesdouble
precisionfloatingpointvalues.
Intheearly1990sBarbiemistakenlysaid,Mathclassistough!Althoughmathisnttough,floatingpoint
arithmeticisntthesameaspencilandpapermath,andcaremustbetaken:
Notallnumberscanberepresentedexactly.
Comparingvaluesshoulduseadelta,insteadofbeingdonedirectlywith==and!=.
Errorsaccumulate.
Operationsarenotalwaystrulyassociativeordistributive.
Increasingtheaccuracycanhelp,butitisnotapanacea.
Often,increasingtheaccuracyandthenroundingtothedesirednumberofdigitsproducesreasonableresults.
UseM(orbignum)toenableMPFRarithmetic.UsePRECtosettheprecisioninbits,andROUNDMODEtosetthe
IEEE754roundingmode.
WithM,gawkperformsarbitraryprecisionintegerarithmeticusingtheGMPlibrary.Thisisfasterandmore
spaceefficientthanusingMPFRforthesamecalculations.
ThereareseveralareaswithrespecttofloatingpointnumberswheregawkdisagreeswiththePOSIXstandard.It
paystobeawareofthem.
Overall,thereisnoneedtobeundulysuspiciousabouttheresultsfromfloatingpointarithmetic.Thelessonto
rememberisthatfloatingpointarithmeticisalwaysmorecomplexthanarithmeticusingpencilandpaper.In
ordertotakeadvantageofthepoweroffloatingpointarithmetic,youneedtoknowitslimitationsandwork
withinthem.Formostcasualuseoffloatingpointarithmetic,youwilloftengettheexpectedresultifyousimply
roundthedisplayofyourfinalresultstothecorrectnumberofsignificantdecimaldigits.
Asgeneraladvice,avoidpresentingnumericaldatainamannerthatimpliesbetterprecisionthanisactuallythe
case.
Next:LanguageHistory,Previous:ArbitraryPrecisionArithmetic,Up:Top[Contents][Index]
ItispossibletoaddnewfunctionswritteninCorC++togawkusingdynamicallyloadedlibraries.Thisfacilityis
availableonsystemsthatsupporttheCdlopen()anddlsym()functions.Thischapterdescribeshowtocreateextensions
300/479
29/7/2016
TheGNUAwkUsersGuide
usingcodewritteninCorC++.
IfyoudontknowanythingaboutCprogramming,youcansafelyskipthischapter,althoughyoumaywishtoreview
thedocumentationontheextensionsthatcomewithgawk(seeExtensionSamples),andtheinformationonthe
gawkextlibproject(seegawkextlib).Thesampleextensionsareautomaticallybuiltandinstalledwhengawkis.
NOTE:Whensandboxisspecified,extensionsaredisabled(seeOptions).
ExtensionIntro:
Whatisanextension.
PluginLicense:
Anoteaboutlicensing.
ExtensionMechanismOutline: Anoutlineofhowitworks.
ExtensionAPIDescription:
AfulldescriptionoftheAPI.
FindingExtensions:
Howgawkfindscompiledextensions.
ExtensionExample:
ExampleCcodeforanextension.
ExtensionSamples:
Thesampleextensionsthatshipwithgawk.
gawkextlib:
Thegawkextlibproject.
Extensionsummary:
Extensionsummary.
ExtensionExercises:
Exercises.
Next:PluginLicense,Up:DynamicExtensions[Contents][Index]
16.1Introduction
Anextension(sometimescalledaplugin)isapieceofexternalcompiledcodethatgawkcanloadatruntimetoprovide
additionalfunctionality,overandabovethebuiltincapabilitiesdescribedintherestofthisWebpage.
Extensionsareusefulbecausetheyallowyou(ofcourse)toextendgawksfunctionality.Forexample,theycanprovide
accesstosystemcalls(suchaschdir()tochangedirectory)andtootherClibraryroutinesthatcouldbeofuse.Aswith
mostsoftware,theskyisthelimitifyoucanimaginesomethingthatyoumightwanttodoandcanwriteinCor
C++,youcanwriteanextensiontodoit!
ExtensionsarewritteninCorC++,usingtheapplicationprogramminginterface(API)definedforthispurposebythe
gawkdevelopers.TherestofthischapterexplainsthefacilitiesthattheAPIprovidesandhowtousethem,andpresents
asmallexampleextension.Inaddition,itdocumentsthesampleextensionsincludedinthegawkdistributionand
describesthegawkextlibproject.SeeExtensionDesign,foradiscussionoftheextensionmechanismgoalsanddesign.
Next:ExtensionMechanismOutline,Previous:ExtensionIntro,Up:DynamicExtensions[Contents][Index]
16.2ExtensionLicensing
EverydynamicextensionmustbedistributedunderalicensethatiscompatiblewiththeGNUGPL(seeCopying).
Inorderfortheextensiontotellgawkthatitisproperlylicensed,theextensionmustdefinetheglobalsymbol
plugin_is_GPL_compatible.Ifthissymboldoesnotexist,gawkemitsafatalerrorandexitswhenittriestoloadyour
extension.
Thedeclaredtypeofthesymbolshouldbeint.Itdoesnotneedtobeinanyallocatedsection,though.Thecodemerely
assertsthatthesymbolexistsintheglobalscope.Somethinglikethisisenough:
intplugin_is_GPL_compatible;
Next:ExtensionAPIDescription,Previous:PluginLicense,Up:DynamicExtensions[Contents][Index]
16.3HowItWorksataHighLevel
301/479
29/7/2016
TheGNUAwkUsersGuide
Communicationbetweengawkandanextensionistwoway.First,whenanextensionisloaded,gawkpassesitapointer
toastructwhosefieldsarefunctionpointers.ThisisshowninFigure16.1.
Figure16.1:Loadingtheextension
Theextensioncancallfunctionsinsidegawkthroughthesefunctionpointers,atruntime,withoutneeding(linktime)
accesstogawkssymbols.Oneofthesefunctionpointersistoafunctionforregisteringnewfunctions.Thisisshown
inFigure16.2.
Figure16.2:Registeringanewfunction
Intheotherdirection,theextensionregistersitsnewfunctionswithgawkbypassingfunctionpointerstothefunctions
thatprovidethenewfeature(do_chdir(),forexample).gawkassociatesthefunctionpointerwithanameandcanthen
callit,usingadefinedcallingconvention.ThisisshowninFigure16.3.
Figure16.3:Callingthenewfunction
Thedo_xxx()function,inturn,thenusesthefunctionpointersintheAPIstructtodoitswork,suchasupdating
variablesorarrays,printingmessages,settingERRNO,andsoon.
302/479
29/7/2016
TheGNUAwkUsersGuide
Conveniencemacrosmakecallingthroughthefunctionpointerslooklikeregularfunctioncallssothatextensioncodeis
quitereadableandunderstandable.
Althoughallofthissoundssomewhatcomplicated,theresultisthatextensioncodeisquitestraightforwardtowriteand
toread.Youcanseethisinthesampleextensionfilefuncs.c(seeExtensionExample)andalsointhetestext.ccode
fortestingtheAPIs.
Someotherbitsandpieces:
TheAPIprovidesaccesstogawksdo_xxxvalues,reflectingcommandlineoptions,likedo_lint,do_profiling,
andsoon(seeExtensionAPIVariables).Theseareinformational:anextensioncannotaffecttheirvaluesinside
gawk.Inaddition,attemptingtoassigntothemproducesacompiletimeerror.
TheAPIalsoprovidesmajorandminorversionnumbers,sothatanextensioncancheckifthegawkitisloaded
withsupportsthefacilitiesitwascompiledwith.(Versionmismatchesshouldnthappen,butweallknowhow
thatgoes.)SeeExtensionVersioning,fordetails.
Next:FindingExtensions,Previous:ExtensionMechanismOutline,Up:DynamicExtensions[Contents][Index]
16.4APIDescription
CorC++codeforanextensionmustincludetheheaderfilegawkapi.h,whichdeclaresthefunctionsanddefinesthe
datatypesusedtocommunicatewithgawk.This(ratherlarge)sectiondescribestheAPIindetail.
ExtensionAPIFunctionsIntroduction: IntroductiontotheAPIfunctions.
GeneralDataTypes:
Thedatatypes.
MemoryAllocationFunctions:
Functionsforallocatingmemory.
ConstructorFunctions:
Functionsforcreatingvalues.
RegistrationFunctions:
Functionstoregisterthingswithgawk.
PrintingMessages:
Functionsforprintingmessages.
UpdatingERRNO:
FunctionsforupdatingERRNO.
RequestingValues:
Howtogetavalue.
AccessingParameters:
Functionsforaccessingparameters.
SymbolTableAccess:
Functionsforaccessingglobalvariables.
ArrayManipulation:
ExtensionAPIVariables:
VariablesprovidedbytheAPI.
ExtensionAPIBoilerplate:
BoilerplatecodeforusingtheAPI.
Next:GeneralDataTypes,Up:ExtensionAPIDescription[Contents][Index]
16.4.1Introduction
Accesstofacilitieswithingawkisachievedbycallingthroughfunctionpointerspassedintoyourextension.
APIfunctionpointersareprovidedforthefollowingkindsofoperations:
Allocating,reallocating,andreleasingmemory.
Registrationfunctions.Youmayregister:
Extensionfunctions
Exitcallbacks
Aversionstring
Inputparsers
Outputwrappers
Twowayprocessors
Allofthesearediscussedindetaillaterinthischapter.
303/479
29/7/2016
TheGNUAwkUsersGuide
Printingfatal,warning,andlintwarningmessages.
UpdatingERRNO,orunsettingit.
Accessingparameters,includingconvertinganundefinedparameterintoanarray.
Symboltableaccess:retrievingaglobalvariable,creatingone,orchangingone.
Creatingandreleasingcachedvaluesthisprovidesanefficientwaytousevaluesformultiplevariablesandcan
beabigperformancewin.
Manipulatingarrays:
Retrieving,adding,deleting,andmodifyingelements
Gettingthecountofelementsinanarray
Creatinganewarray
Clearinganarray
FlatteninganarrayforeasyCstyleloopingoverallitsindicesandelements
SomepointsaboutusingtheAPI:
Thefollowingtypes,macros,and/orfunctionsarereferencedingawkapi.h.Forcorrectuse,youmusttherefore
includethecorrespondingstandardheaderfilebeforeincludinggawkapi.h:
Centity
Headerfile
EOF
<stdio.h>
Valuesforerrno <errno.h>
FILE
NULL
memcpy()
memset()
size_t
structstat
<stdio.h>
<stddef.h>
<string.h>
<string.h>
<sys/types.h>
<sys/stat.h>
Duetoportabilityconcerns,especiallytosystemsthatarenotfullystandardscompliant,itisyourresponsibility
toincludethecorrectfilesinthecorrectway.Thisrequirementisnecessaryinordertokeepgawkapi.hclean,
insteadofbecomingaportabilityhodgepodgeascanbeseeninsomepartsofthegawksourcecode.
Thegawkapi.hfilemaybeincludedmorethanoncewithoutilleffect.Doingso,however,ispoorcodingpractice.
AlthoughtheAPIonlyusesISOC90features,thereisanexceptiontheconstructorfunctionsusetheinline
keyword.Ifyourcompilerdoesnotsupportthiskeyword,youshouldeitherplaceDinline=''onyour
commandlineorusetheGNUAutotoolsandincludeaconfig.hfileinyourextensions.
Allpointersfilledinbygawkpointtomemorymanagedbygawkandshouldbetreatedbytheextensionasread
only.Memoryforallstringspassedintogawkfromtheextensionmustcomefromcallingoneofgawk_malloc(),
gawk_calloc(),orgawk_realloc(),andismanagedbygawkfromthenon.
TheAPIdefinesseveralsimplestructsthatmapvaluesasseenfromawk.Avaluecanbeadouble,astring,oran
array(asinmultidimensionalarrays,orwhencreatinganewarray).Stringvaluesmaintainbothpointerand
length,becauseembeddedNULcharactersareallowed.
NOTE:Byintent,stringsaremaintainedusingthecurrentmultibyteencoding(asdefinedbyLC_xxx
environmentvariables)andnotusingwidecharacters.Thismatcheshowgawkstoresstrings
internallyandalsohowcharactersarelikelytobeinputintoandoutputfromfiles.
Whenretrievingavalue(suchasaparameterorthatofaglobalvariableorarrayelement),theextensionrequests
aspecifictype(number,string,scalar,valuecookie,array,orundefined).Whentherequestisundefined,the
returnedvaluewillhavetherealunderlyingtype.
However,iftherequestandactualtypedontmatch,theaccessfunctionreturnsfalseandfillsinthetypeofthe
actualvaluethatisthere,sothattheextensioncan,e.g.,printanerrormessage(suchasscalarpassedwhere
arrayexpected).
YoumaycalltheAPIfunctionsbyusingthefunctionpointersdirectly,buttheinterfaceisnotsopretty.Tomake
extensioncodelookmorelikeregularcode,thegawkapi.hheaderfiledefinesseveralmacrosthatyoushoulduseinyour
code.Thissectionpresentsthemacrosasiftheywerefunctions.
304/479
29/7/2016
TheGNUAwkUsersGuide
Next:MemoryAllocationFunctions,Previous:ExtensionAPIFunctionsIntroduction,Up:ExtensionAPIDescription
[Contents][Index]
16.4.2GeneralPurposeDataTypes
Ihaveatruelove/haterelationshipwithunions.
ArnoldRobbins
Thatsthethingaboutunions:thecompilerwillarrangethingssotheycanaccommodatebothloveand
hate.
ChetRamey
TheextensionAPIdefinesanumberofsimpletypesandstructuresforgeneralpurposeuse.Additional,more
specialized,datastructuresareintroducedinsubsequentsections,togetherwiththefunctionsthatusethem.
Thegeneralpurposetypesandstructuresareasfollows:
typedefvoid*awk_ext_id_t;
Avalueofthistypeisreceivedfromgawkwhenanextensionisloaded.Thatvaluemustthenbepassedbackto
gawkasthefirstparameterofeachAPIfunction.
#defineawk_const
Thismacroexpandstoconstwhencompilinganextension,andtonothingwhencompilinggawkitself.This
makescertainfieldsintheAPIdatastructuresunwritablefromextensioncode,whileallowinggawktousethem
asitneedsto.
typedefenumawk_bool{
awk_false=0,
awk_true
}awk_bool_t;
AsimpleBooleantype.
typedefstructawk_string{
char*str;/*data*/
size_tlen;/*lengththereof,inchars*/
}awk_string_t;
Thisrepresentsamutablestring.gawkownsthememorypointedtoifitsuppliedthevalue.Otherwise,ittakes
ownershipofthememorypointedto.Suchmemorymustcomefromcallingoneofthegawk_malloc(),
gawk_calloc(),orgawk_realloc()functions!
Asmentionedearlier,stringsaremaintainedusingthecurrentmultibyteencoding.
typedefenum{
AWK_UNDEFINED,
AWK_NUMBER,
AWK_STRING,
AWK_ARRAY,
AWK_SCALAR,/*opaqueaccesstoavariable*/
AWK_VALUE_COOKIE/*forupdatingapreviouslycreatedvalue*/
}awk_valtype_t;
Thisenumindicatesthetypeofavalue.Itisusedinthefollowingstruct.
typedefstructawk_value{
awk_valtype_tval_type;
305/479
29/7/2016
TheGNUAwkUsersGuide
union{
awk_string_ts;
doubled;
awk_array_ta;
awk_scalar_tscl;
awk_value_cookie_tvc;
}u;
}awk_value_t;
Anawkvalue.Theval_typememberindicateswhatkindofvaluetheunionholds,andeachmemberisofthe
appropriatetype.
#definestr_valueu.s
#definenum_valueu.d
#definearray_cookieu.a
#definescalar_cookieu.scl
#definevalue_cookieu.vc
Usingthesemacrosmakesaccessingthefieldsoftheawk_value_tmorereadable.
typedefvoid*awk_scalar_t;
Scalarscanberepresentedasanopaquetype.Thesevaluesareobtainedfromgawkandthenpassedbackintoit.
Thisisdiscussedinageneralfashioninthetextfollowingthislist,andinmoredetailinSymboltablebycookie.
typedefvoid*awk_value_cookie_t;
Avaluecookieisanopaquetyperepresentingacachedvalue.Thisisalsodiscussedinageneralfashioninthe
textfollowingthislist,andinmoredetailinCachedvalues.
Scalarvaluesinawkareeithernumbersorstrings.Theawk_value_tstructrepresentsvalues.Theval_typemember
indicateswhatisintheunion.
RepresentingnumbersiseasytheAPIusesaCdouble.Stringsrequiremorework.Becausegawkallowsembedded
NULbytesinstringvalues,astringmustberepresentedasapaircontainingadatapointerandlength.Thisisthe
awk_string_ttype.
Identifiers(i.e.,thenamesofglobalvariables)canbeassociatedwitheitherscalarvaluesorwitharrays.Inaddition,
gawkprovidestruearraysofarrays,whereanygivenarrayelementcanitselfbeanarray.Discussionofarraysisdelayed
untilArrayManipulation.
Thevariousmacroslistedearliermakeiteasiertousetheelementsoftheunionasiftheywerefieldsinastructthisis
acommoncodingpracticeinC.Suchcodeiseasiertowriteandtoread,butitremainsyourresponsibilitytomakesure
thattheval_typemembercorrectlyreflectsthetypeofthevalueintheawk_value_tstruct.
Conceptually,thefirstthreemembersoftheunion(number,string,andarray)areallthatisneededforworkingwithawk
values.However,becausetheAPIprovidesroutinesforaccessingandchangingthevalueofaglobalscalarvariable
onlybyusingthevariablesname,thereisaperformancepenalty:gawkmustfindthevariableeachtimeitisaccessed
andchanged.Thisturnsouttobearealissue,notjustatheoreticalone.
Thus,ifyouknowthatyourextensionwillspendconsiderabletimereadingand/orchangingthevalueofoneormore
scalarvariables,youcanobtainascalarcookie98objectforthatvariable,andthenusethecookieforgettingthe
variablesvalueorforchangingthevariablesvalue.Theawk_scalar_ttypeholdsascalarcookie,andthe
scalar_cookiemacroprovidesaccesstothevalueofthattypeintheawk_value_tstruct.Givenascalarcookie,gawkcan
directlyretrieveormodifythevalue,asrequired,withouthavingtofinditfirst.
Theawk_value_cookie_ttypeandvalue_cookiemacroaresimilar.Ifyouknowthatyouwishtousethesamenumeric
orstringvalueforoneormorevariables,youcancreatethevalueonce,retainingavaluecookieforit,andthenpassin
thatvaluecookiewheneveryouwishtosetthevalueofavariable.Thissavesstoragespacewithintherunninggawk
processandreducesthetimeneededtocreatethevalue.
306/479
29/7/2016
TheGNUAwkUsersGuide
Next:ConstructorFunctions,Previous:GeneralDataTypes,Up:ExtensionAPIDescription[Contents][Index]
16.4.3MemoryAllocationFunctionsandConvenienceMacros
TheAPIprovidesanumberofmemoryallocationfunctionsforallocatingmemorythatcanbepassedtogawk,aswellas
anumberofconveniencemacros.Thissubsectionpresentsthemallasfunctionprototypes,inthewaythatextension
codewouldusethem:
void*gawk_malloc(size_tsize);
Callthecorrectversionofmalloc()toallocatestoragethatmaybepassedtogawk.
void*gawk_calloc(size_tnmemb,size_tsize);
Callthecorrectversionofcalloc()toallocatestoragethatmaybepassedtogawk.
void*gawk_realloc(void*ptr,size_tsize);
Callthecorrectversionofrealloc()toallocatestoragethatmaybepassedtogawk.
voidgawk_free(void*ptr);
Callthecorrectversionoffree()toreleasestoragethatwasallocatedwithgawk_malloc(),gawk_calloc(),or
gawk_realloc().
TheAPIhastoprovidethesefunctionsbecauseitispossibleforanextensiontobecompiledandlinkedagainsta
differentversionoftheClibrarythanwasusedforthegawkexecutable.99Ifgawkweretouseitsversionoffree()when
thememorycamefromanunrelatedversionofmalloc(),unexpectedbehaviorwouldlikelyresult.
Twoconveniencemacrosmaybeusedforallocatingstoragefromgawk_malloc()andgawk_realloc().Iftheallocation
fails,theycausegawktoexitwithafatalerrormessage.Theyshouldbeusedasiftheywereprocedurecallsthatdonot
returnavalue:
#defineemalloc(pointer,type,size,message)
Theargumentstothismacroareasfollows:
pointer
Thepointervariabletopointattheallocatedstorage.
type
Thetypeofthepointervariable.Thisisusedtocreateacastforthecalltogawk_malloc().
size
Thetotalnumberofbytestobeallocated.
message
Amessagetobeprefixedtothefatalerrormessage.Typicallythisisthenameofthefunctionusingthe
macro.
Forexample,youmightallocateastringvaluelikeso:
awk_value_tresult;
char*message;
constchargreet[]="Don'tPanic!";
emalloc(message,char*,sizeof(greet),"myfunc");
strcpy(message,greet);
307/479
29/7/2016
TheGNUAwkUsersGuide
make_malloced_string(message,strlen(message),&result);
#defineerealloc(pointer,type,size,message)
Thisislikeemalloc(),butitcallsgawk_realloc()insteadofgawk_malloc().Theargumentsarethesameasfor
theemalloc()macro.
Next:RegistrationFunctions,Previous:MemoryAllocationFunctions,Up:ExtensionAPIDescription[Contents]
[Index]
16.4.4ConstructorFunctions
TheAPIprovidesanumberofconstructorfunctionsforcreatingstringandnumericvalues,aswellasanumberof
conveniencemacros.Thissubsectionpresentsthemallasfunctionprototypes,inthewaythatextensioncodewoulduse
them:
staticinlineawk_value_t*
make_const_string(constchar*string,size_tlength,awk_value_t*result);
Thisfunctioncreatesastringvalueintheawk_value_tvariablepointedtobyresult.ItexpectsstringtobeaC
stringconstant(orotherstringdata),andautomaticallycreatesacopyofthedataforstorageinresult.Itreturns
result.
make_malloced_string(constchar*string,size_tlength,awk_value_t*result);
Thisfunctioncreatesastringvalueintheawk_value_tvariablepointedtobyresult.Itexpectsstringtobea
char*valuepointingtodatapreviouslyobtainedfrom gawk_malloc(),gawk_calloc(),orgawk_realloc().The
ideahereisthatthedataispasseddirectlytogawk,whichassumesresponsibilityforit.Itreturnsresult.
make_null_string(awk_value_t*result);
Thisspecializedfunctioncreatesanullstring(theundefinedvalue)intheawk_value_tvariablepointedtoby
result.Itreturnsresult.
make_number(doublenum,awk_value_t*result);
Thisfunctionsimplycreatesanumericvalueintheawk_value_tvariablepointedtobyresult.
Next:PrintingMessages,Previous:ConstructorFunctions,Up:ExtensionAPIDescription[Contents][Index]
16.4.5RegistrationFunctions
ThissectiondescribestheAPIfunctionsforregisteringpartsofyourextensionwithgawk.
ExtensionFunctions:
Registeringextensionfunctions.
ExitCallbackFunctions: Registeringanexitcallback.
ExtensionVersionString: Registeringaversionstring.
InputParsers:
Registeringaninputparser.
OutputWrappers:
Registeringanoutputwrapper.
Twowayprocessors:
Registeringatwowayprocessor.
Next:ExitCallbackFunctions,Up:RegistrationFunctions[Contents][Index]
16.4.5.1RegisteringAnExtensionFunction
308/479
29/7/2016
TheGNUAwkUsersGuide
Extensionfunctionsaredescribedbythefollowingrecord:
typedefstructawk_ext_func{
constchar*name;
awk_value_t*(*function)(intnum_actual_args,awk_value_t*result);
size_tnum_expected_args;
}awk_ext_func_t;
Thefieldsare:
constchar*name;
Thenameofthenewfunction.awklevelcodecallsthefunctionbythisname.ThisisaregularCstring.
Functionnamesmustobeytherulesforawkidentifiers.Thatis,theymustbeginwitheitheranEnglishletteroran
underscore,whichmaybefollowedbyanynumberofletters,digits,andunderscores.Lettercaseinfunction
namesissignificant.
awk_value_t*(*function)(intnum_actual_args,awk_value_t*result);
ThisisapointertotheCfunctionthatprovidestheextensionsfunctionality.Thefunctionmustfillin*result
witheitheranumberorastring.gawktakesownershipofanystringmemory.Asmentionedearlier,stringmemory
mustcomefromoneofgawk_malloc(),gawk_calloc(),orgawk_realloc().
Thenum_actual_argsargumenttellstheCfunctionhowmanyactualparameterswerepassedfromthecallingawk
code.
Thefunctionmustreturnthevalueofresult.Thisisfortheconvenienceofthecallingcodeinsidegawk.
size_tnum_expected_args;
Thisisthenumberofargumentsthefunctionexpectstoreceive.Eachextensionfunctionmaydecidewhattodoif
thenumberofargumentsisntwhatitexpected.Aswithrealawkfunctions,itislikelyOKtoignoreextra
arguments.
Onceyouhavearecordrepresentingyourextensionfunction,youregisteritwithgawkusingthisAPIfunction:
awk_bool_tadd_ext_func(constchar*namespace,constawk_ext_func_t*func);
Thisfunctionreturnstrueuponsuccess,falseotherwise.Thenamespaceparameteriscurrentlynotusedyou
shouldpassinanemptystring("").Thefuncpointeristheaddressofastructrepresentingyourfunction,asjust
described.
Next:ExtensionVersionString,Previous:ExtensionFunctions,Up:RegistrationFunctions[Contents][Index]
16.4.5.2RegisteringAnExitCallbackFunction
Anexitcallbackfunctionisafunctionthatgawkcallsbeforeitexits.Suchfunctionsareusefulifyouhavegeneral
cleanuptasksthatshouldbeperformedinyourextension(suchasclosingdatabaseconnectionsorotherresource
deallocations).Youcanregistersuchafunctionwithgawkusingthefollowingfunction:
voidawk_atexit(void(*funcp)(void*data,intexit_status),
void*arg0);
Theparametersare:
funcp
Apointertothefunctiontobecalledbeforegawkexits.Thedataparameterwillbetheoriginalvalueof
arg0.Theexit_statusparameteristheexitstatusvaluethatgawkintendstopasstotheexit()systemcall.
309/479
29/7/2016
TheGNUAwkUsersGuide
arg0
Apointertoprivatedatathatgawksavesinordertopasstothefunctionpointedtobyfuncp.
Exitcallbackfunctionsarecalledinlastin,firstout(LIFO)orderthatis,inthereverseorderinwhichtheyare
registeredwithgawk.
Next:InputParsers,Previous:ExitCallbackFunctions,Up:RegistrationFunctions[Contents][Index]
16.4.5.3RegisteringAnExtensionVersionString
Youcanregisteraversionstringthatindicatesthenameandversionofyourextensionwithgawk,asfollows:
voidregister_ext_version(constchar*version);
Registerthestringpointedtobyversionwithgawk.Notethatgawkdoesnotcopytheversionstring,soitshould
notbechanged.
gawkprintsallregisteredextensionversionstringswhenitisinvokedwiththeversionoption.
Next:OutputWrappers,Previous:ExtensionVersionString,Up:RegistrationFunctions[Contents][Index]
16.4.5.4CustomizedInputParsers
Bydefault,gawkreadstextfilesasitsinput.ItusesthevalueofRStofindtheendoftherecord,andthenusesFS(or
FIELDWIDTHSorFPAT)tosplititintofields(seeReadingFiles).Additionally,itsetsthevalueofRT(seeBuiltin
Variables).
Ifyouwant,youcanprovideyourowncustominputparser.Aninputparsersjobistoreturnarecordtothegawk
recordprocessingcode,alongwithindicatorsforthevalueandlengthofthedatatobeusedforRT,ifany.
Toprovideaninputparser,youmustfirstprovidetwofunctions(whereXXXisaprefixnameforyourextension):
awk_bool_tXXX_can_take_file(constawk_input_buf_t*iobuf);
Thisfunctionexaminestheinformationavailableiniobuf(whichwediscussshortly).Basedontheinformation
there,itdecidesiftheinputparsershouldbeusedforthisfile.Ifso,itshouldreturntrue.Otherwise,itshould
returnfalse.Itshouldnotchangeanystate(variablevalues,etc.)withingawk.
awk_bool_tXXX_take_control_of(awk_input_buf_t*iobuf);
Whengawkdecidestohandcontrolofthefileovertotheinputparser,itcallsthisfunction.Thisfunctioninturn
mustfillincertainfieldsintheawk_input_buf_tstructureandensurethatcertainconditionsaretrue.Itshould
thenreturntrue.Ifanerrorofsomekindoccurs,itshouldnotfillinanyfieldsandshouldreturnfalsethengawk
willnotusetheinputparser.Thedetailsarepresentedshortly.
Yourextensionshouldpackagethesefunctionsinsideanawk_input_parser_t,whichlookslikethis:
typedefstructawk_input_parser{
constchar*name;/*nameofparser*/
awk_bool_t(*can_take_file)(constawk_input_buf_t*iobuf);
awk_bool_t(*take_control_of)(awk_input_buf_t*iobuf);
awk_conststructawk_input_parser*awk_constnext;/*forgawk*/
}awk_input_parser_t;
Thefieldsare:
constchar*name;
310/479
29/7/2016
TheGNUAwkUsersGuide
Thenameoftheinputparser.ThisisaregularCstring.
awk_bool_t(*can_take_file)(constawk_input_buf_t*iobuf);
ApointertoyourXXX_can_take_file()function.
awk_bool_t(*take_control_of)(awk_input_buf_t*iobuf);
ApointertoyourXXX_take_control_of()function.
awk_conststructinput_parser*awk_constnext;
Thisisforusebygawkthereforeitismarkedawk_constsothattheextensioncannotmodifyit.
Thestepsareasfollows:
1.Createastaticawk_input_parser_tvariableandinitializeitappropriately.
2.Whenyourextensionisloaded,registeryourinputparserwithgawkusingtheregister_input_parser()API
function(describednext).
Anawk_input_buf_tlookslikethis:
typedefstructawk_input{
constchar*name;/*filename*/
intfd;/*filedescriptor*/
#defineINVALID_HANDLE(1)
void*opaque;/*privatedataforinputparsers*/
int(*get_record)(char**out,structawk_input*iobuf,
int*errcode,char**rt_start,size_t*rt_len);
ssize_t(*read_func)();
void(*close_func)(structawk_input*iobuf);
structstatsbuf;/*statbuf*/
}awk_input_buf_t;
Thefieldscanbedividedintotwocategories:thoseforuse(initially,atleast)byXXX_can_take_file(),andthoseforuse
byXXX_take_control_of().Thefirstgroupoffieldsandtheirusesareasfollows:
constchar*name;
Thenameofthefile.
intfd;
Afiledescriptorforthefile.Ifgawkwasabletoopenthefile,thenfdwillnotbeequaltoINVALID_HANDLE.
Otherwise,itwill.
structstatsbuf;
Ifthefiledescriptorisvalid,thengawkwillhavefilledinthisstructureviaacalltothefstat()systemcall.
TheXXX_can_take_file()functionshouldexaminethesefieldsanddecideiftheinputparsershouldbeusedforthefile.
Thedecisioncanbemadebasedupongawkstate(thevalueofavariabledefinedpreviouslybytheextensionandsetby
awkcode),thenameofthefile,whetherornotthefiledescriptorisvalid,theinformationinthestructstat,orany
combinationofthesefactors.
OnceXXX_can_take_file()hasreturnedtrue,andgawkhasdecidedtouseyourinputparser,itcalls
XXX_take_control_of().Thatfunctionthenfillseithertheget_recordfieldortheread_funcfieldinthe
awk_input_buf_t.ItmustalsoensurethatfdisnotsettoINVALID_HANDLE.Thefollowinglistdescribesthefieldsthat
maybefilledbyXXX_take_control_of():
void*opaque;
Thisisusedtoholdanystateinformationneededbytheinputparserforthisfile.Itisopaquetogawk.Theinput
311/479
29/7/2016
TheGNUAwkUsersGuide
parserisnotrequiredtousethispointer.
int(*get_record)(char**out,
structawk_input*iobuf,
int*errcode,
char**rt_start,
size_t*rt_len);
Thisfunctionpointershouldpointtoafunctionthatcreatestheinputrecords.Saidfunctionisthecoreofthe
inputparser.Itsbehaviorisdescribedinthetextfollowingthislist.
ssize_t(*read_func)();
ThisfunctionpointershouldpointtoafunctionthathasthesamebehaviorasthestandardPOSIXread()system
call.Itisanalternativetotheget_recordpointer.Itsbehaviorisalsodescribedinthetextfollowingthislist.
void(*close_func)(structawk_input*iobuf);
Thisfunctionpointershouldpointtoafunctionthatdoestheteardown.Itshouldreleaseanyresources
allocatedbyXXX_take_control_of().Itmayalsoclosethefile.Ifitdoesso,itshouldsetthefdfieldto
INVALID_HANDLE.
IffdisstillnotINVALID_HANDLEafterthecalltothisfunction,gawkcallstheregularclose()systemcall.
Havingateardownfunctionisoptional.Ifyourinputparserdoesnotneedit,donotsetthisfield.Then,gawk
callstheregularclose()systemcallonthefiledescriptor,soitshouldbevalid.
TheXXX_get_record()functiondoestheworkofcreatinginputrecords.Theparametersareasfollows:
char**out
Thisisapointertoachar*variablethatissettopointtotherecord.gawkmakesitsowncopyofthedata,sothe
extensionmustmanagethisstorage.
structawk_input*iobuf
Thisistheawk_input_buf_tforthefile.Thefieldsshouldbeusedforreadingdata(fd)andformanagingprivate
state(opaque),ifany.
int*errcode
Ifanerroroccurs,*errcodeshouldbesettoanappropriatecodefrom<errno.h>.
char**rt_start
size_t*rt_len
Iftheconceptofarecordterminatormakessense,then*rt_startshouldbesettopointtothedatatobeused
forRT,and*rt_lenshouldbesettothelengthofthedata.Otherwise,*rt_lenshouldbesettozero.gawkmakes
itsowncopyofthisdata,sotheextensionmustmanagethisstorage.
Thereturnvalueisthelengthofthebufferpointedtoby*out,orEOFifendoffilewasreachedoranerroroccurred.
Itisguaranteedthaterrcodeisavalidpointer,sothereisnoneedtotestforaNULLvalue.gawksets*errcodetozero,so
thereisnoneedtosetitunlessanerroroccurs.
Ifanerrordoesoccur,thefunctionshouldreturnEOFandset*errcodetoavaluegreaterthanzero.Inthatcase,if
*errcodedoesnotequalzero,gawkautomaticallyupdatestheERRNOvariablebasedonthevalueof*errcode.(Ingeneral,
setting*errcode=errnoshoulddotherightthing.)
Asanalternativetosupplyingafunctionthatreturnsaninputrecord,youmayinsteadsupplyafunctionthatsimply
readsbytes,andletgawkparsethedataintorecords.Ifyoudoso,thedatashouldbereturnedinthemultibyteencoding
ofthecurrentlocale.Suchafunctionshouldfollowthesamebehaviorastheread()systemcall,andyoufillinthe
312/479
29/7/2016
TheGNUAwkUsersGuide
read_funcpointerwithitsaddressintheawk_input_buf_tstructure.
Bydefault,gawksetstheread_funcpointertopointtotheread()systemcall.Soyourextensionneednotsetthisfield
explicitly.
NOTE:Youmustchooseonemethodortheother:eitherafunctionthatreturnsarecord,oronethatreturns
rawdata.Inparticular,ifyousupplyafunctiontogetarecord,gawkwillcallit,andwillnevercalltheraw
readfunction.
gawkshipswithasampleextensionthatreadsdirectories,returningrecordsforeachentryinadirectory(seeExtension
SampleReaddir).Youmaywishtousethatcodeasaguideforwritingyourowninputparser.
Whenwritinganinputparser,youshouldthinkabout(anddocument)howitisexpectedtointeractwithawkcode.You
maywantittoalwaysbecalled,andtotakeeffectasappropriate(asthereaddirextensiondoes).Oryoumaywantitto
takeeffectbaseduponthevalueofanawkvariable,astheXMLextensionfromthegawkextlibprojectdoes(see
gawkextlib).Inthelattercase,codeinaBEGINFILEsectioncanlookatFILENAMEandERRNOtodecidewhetherornotto
activateaninputparser(seeBEGINFILE/ENDFILE).
Youregisteryourinputparserwiththefollowingfunction:
voidregister_input_parser(awk_input_parser_t*input_parser);
Registertheinputparserpointedtobyinput_parserwithgawk.
Next:Twowayprocessors,Previous:InputParsers,Up:RegistrationFunctions[Contents][Index]
16.4.5.5CustomizedOutputWrappers
Anoutputwrapperisthemirrorimageofaninputparser.Itallowsanextensiontotakeovertheoutputtoafileopened
withthe>or>>I/Oredirectionoperators(seeRedirection).
Theoutputwrapperisverysimilartotheinputparserstructure:
typedefstructawk_output_wrapper{
constchar*name;/*nameofthewrapper*/
awk_bool_t(*can_take_file)(constawk_output_buf_t*outbuf);
awk_bool_t(*take_control_of)(awk_output_buf_t*outbuf);
awk_conststructawk_output_wrapper*awk_constnext;/*forgawk*/
}awk_output_wrapper_t;
Themembersareasfollows:
constchar*name;
Thisisthenameoftheoutputwrapper.
awk_bool_t(*can_take_file)(constawk_output_buf_t*outbuf);
Thispointstoafunctionthatexaminestheinformationintheawk_output_buf_tstructurepointedtobyoutbuf.It
shouldreturntrueiftheoutputwrapperwantstotakeoverthefile,andfalseotherwise.Itshouldnotchangeany
state(variablevalues,etc.)withingawk.
awk_bool_t(*take_control_of)(awk_output_buf_t*outbuf);
Thefunctionpointedtobythisfieldiscalledwhengawkdecidestolettheoutputwrappertakecontrolofthefile.
Itshouldfillinappropriatemembersoftheawk_output_buf_tstructure,asdescribednext,andreturntrueif
successful,falseotherwise.
awk_conststructoutput_wrapper*awk_constnext;
313/479
29/7/2016
TheGNUAwkUsersGuide
Theawk_output_buf_tstructurelookslikethis:
typedefstructawk_output_buf{
constchar*name;/*nameofoutputfile*/
constchar*mode;/*modeargumenttofopen*/
FILE*fp;/*stdiofilepointer*/
awk_bool_tredirected;/*trueifawrapperisactive*/
void*opaque;/*forusebyoutputwrapper*/
size_t(*gawk_fwrite)(constvoid*buf,size_tsize,size_tcount,
FILE*fp,void*opaque);
int(*gawk_fflush)(FILE*fp,void*opaque);
int(*gawk_ferror)(FILE*fp,void*opaque);
int(*gawk_fclose)(FILE*fp,void*opaque);
}awk_output_buf_t;
Heretoo,yourextensionwilldefineXXX_can_take_file()andXXX_take_control_of()functionsthatexamineand
updatedatamembersintheawk_output_buf_t.Thedatamembersareasfollows:
constchar*name;
Thenameoftheoutputfile.
constchar*mode;
Themodestring(aswouldbeusedinthesecondargumenttofopen())withwhichthefilewasopened.
FILE*fp;
TheFILEpointerfrom<stdio.h>.gawkopensthefilebeforeattemptingtofindanoutputwrapper.
awk_bool_tredirected;
ThisfieldmustbesettotruebytheXXX_take_control_of()function.
void*opaque;
Thispointerisopaquetogawk.Theextensionshoulduseittostoreapointertoanyprivatedataassociatedwith
thefile.
size_t(*gawk_fwrite)(constvoid*buf,size_tsize,size_tcount,
FILE*fp,void*opaque);
int(*gawk_fflush)(FILE*fp,void*opaque);
int(*gawk_ferror)(FILE*fp,void*opaque);
int(*gawk_fclose)(FILE*fp,void*opaque);
Thesepointersshouldbesettopointtofunctionsthatperformtheequivalentfunctionasthe<stdio.h>functions
do,ifappropriate.gawkusesthesefunctionpointersforalloutput.gawkinitializesthepointerstopointtointernal
passthroughfunctionsthatjustcalltheregular<stdio.h>functions,soanextensiononlyneedstoredefine
thosefunctionsthatareappropriateforwhatitdoes.
TheXXX_can_take_file()functionshouldmakeadecisionbaseduponthenameandmodefields,andanyadditionalstate
(suchasawkvariablevalues)thatisappropriate.
WhengawkcallsXXX_take_control_of(),thatfunctionshouldfillintheotherfieldsasappropriate,exceptforfp,which
itshouldjustusenormally.
Youregisteryouroutputwrapperwiththefollowingfunction:
voidregister_output_wrapper(awk_output_wrapper_t*output_wrapper);
Registertheoutputwrapperpointedtobyoutput_wrapperwithgawk.
314/479
29/7/2016
TheGNUAwkUsersGuide
Previous:OutputWrappers,Up:RegistrationFunctions[Contents][Index]
16.4.5.6CustomizedTwowayProcessors
AtwowayprocessorcombinesaninputparserandanoutputwrapperfortwowayI/Owiththe|&operator(see
Redirection).Itmakesidenticaluseoftheawk_input_parser_tandawk_output_buf_tstructuresasdescribedearlier.
Atwowayprocessorisrepresentedbythefollowingstructure:
typedefstructawk_two_way_processor{
constchar*name;/*nameofthetwowayprocessor*/
awk_bool_t(*can_take_two_way)(constchar*name);
awk_bool_t(*take_control_of)(constchar*name,
awk_input_buf_t*inbuf,
awk_output_buf_t*outbuf);
awk_conststructawk_two_way_processor*awk_constnext;/*forgawk*/
}awk_two_way_processor_t;
Thefieldsareasfollows:
constchar*name;
Thenameofthetwowayprocessor.
awk_bool_t(*can_take_two_way)(constchar*name);
ThefunctionpointedtobythisfieldshouldreturntrueifitwantstotakeovertwowayI/Oforthisfilename.It
shouldnotchangeanystate(variablevalues,etc.)withingawk.
awk_bool_t(*take_control_of)(constchar*name,
awk_input_buf_t*inbuf,
awk_output_buf_t*outbuf);
Thefunctionpointedtobythisfieldshouldfillintheawk_input_buf_tandawk_outut_buf_tstructurespointedto
byinbufandoutbuf,respectively.Thesestructuresweredescribedearlier.
awk_conststructtwo_way_processor*awk_constnext;
Aswiththeinputparserandoutputprocessor,youprovideyesIcantakethisandtakeoverforthisfunctions,
XXX_can_take_two_way()andXXX_take_control_of().
Youregisteryourtwowayprocessorwiththefollowingfunction:
voidregister_two_way_processor(awk_two_way_processor_t*two_way_processor);
Registerthetwowayprocessorpointedtobytwo_way_processorwithgawk.
Next:UpdatingERRNO,Previous:RegistrationFunctions,Up:ExtensionAPIDescription[Contents][Index]
16.4.6PrintingMessages
Youcanprintdifferentkindsofwarningmessagesfromyourextension,asdescribedhere.Notethatforthesefunctions,
youmustpassintheextensionIDreceivedfromgawkwhentheextensionwasloaded:100
voidfatal(awk_ext_id_tid,constchar*format,...);
Printamessageandthencausegawktoexitimmediately.
315/479
29/7/2016
TheGNUAwkUsersGuide
voidwarning(awk_ext_id_tid,constchar*format,...);
Printawarningmessage.
voidlintwarn(awk_ext_id_tid,constchar*format,...);
Printalintwarning.Normallythisisthesameasprintingawarningmessage,butifgawkwasinvokedwith
lint=fatal,thenlintwarningsbecomefatalerrormessages.
AllofthesefunctionsareotherwiseliketheCprintf()familyoffunctions,wheretheformatparameterisastringwith
literalcharactersandformattingcodesintermixed.
Next:RequestingValues,Previous:PrintingMessages,Up:ExtensionAPIDescription[Contents][Index]
16.4.7UpdatingERRNO
ThefollowingfunctionsallowyoutoupdatetheERRNOvariable:
voidupdate_ERRNO_int(interrno_val);
SetERRNOtothestringequivalentoftheerrorcodeinerrno_val.Thevalueshouldbeoneofthedefinederror
codesin<errno.h>,andgawkturnsitintoa(possiblytranslated)stringusingtheCstrerror()function.
voidupdate_ERRNO_string(constchar*string);
SetERRNOdirectlytothestringvalueofERRNO.gawkmakesacopyofthevalueofstring.
voidunset_ERRNO(void);
UnsetERRNO.
Next:AccessingParameters,Previous:UpdatingERRNO,Up:ExtensionAPIDescription[Contents][Index]
16.4.8RequestingValues
Allofthefunctionsthatreturnvaluesfromgawkworkinthesameway.Youpassinanawk_valtype_tvaluetoindicate
whatkindofvalueyouexpect.Iftheactualvaluematcheswhatyourequested,thefunctionreturnstrueandfillsinthe
awk_value_tresult.Otherwise,thefunctionreturnsfalse,andtheval_typememberindicatesthetypeoftheactual
value.Youmaythenprintanerrormessageorreissuetherequestfortheactualvaluetype,asappropriate.Thisbehavior
issummarizedinTable16.1.
TypeofActualValue
String
String
Number
Type
Requested
Array
Scalar
Undefined
Valuecookie
String
Numberifcanbe
converted,elsefalse
False
Scalar
String
False
Number
String
Array
False
Undefined
False
Number
False
False
False
Scalar
Number
False
Array
False
Array
False
False
False
Undefined
False
Table16.1:APIvaluetypesreturned
Next:SymbolTableAccess,Previous:RequestingValues,Up:ExtensionAPIDescription[Contents][Index]
316/479
29/7/2016
TheGNUAwkUsersGuide
16.4.9AccessingandUpdatingParameters
Twofunctionsgiveyouaccesstothearguments(parameters)passedtoyourextensionfunction.Theyare:
awk_bool_tget_argument(size_tcount,
awk_valtype_twanted,
awk_value_t*result);
Fillintheawk_value_tstructurepointedtobyresultwiththecountthargument.Returntrueiftheactualtype
matcheswanted,andfalseotherwise.Inthelattercase,result>val_typeindicatestheactualtype(seeTable
16.1).Countsarezerobasedthefirstargumentisnumberedzero,thesecondone,andsoon.wantedindicates
thetypeofvalueexpected.
awk_bool_tset_argument(size_tcount,awk_array_tarray);
Convertaparameterthatwasundefinedintoanarraythisprovidescallbyreferenceforarrays.Returnfalseif
countistoobig,oriftheargumentstypeisnotundefined.SeeArrayManipulation,formoreinformationon
creatingarrays.
Next:ArrayManipulation,Previous:AccessingParameters,Up:ExtensionAPIDescription[Contents][Index]
16.4.10SymbolTableAccess
Twosetsofroutinesprovideaccesstoglobalvariables,andonesetallowsyoutocreateandreleasecachedvalues.
Symboltablebyname: Accessingvariablesbyname.
Symboltablebycookie: Accessingvariablesbycookie.
Cachedvalues:
Creatingandusingcachedvalues.
Next:Symboltablebycookie,Up:SymbolTableAccess[Contents][Index]
16.4.10.1VariableAccessandUpdatebyName
Thefollowingroutinesprovidetheabilitytoaccessandupdateglobalawklevelvariablesbyname.Incompiler
terminology,identifiersofdifferentkindsaretermedsymbols,thusthesymintheroutinesnames.Thedatastructure
thatstoresinformationaboutsymbolsistermedasymboltable.Thefunctionsareasfollows:
awk_bool_tsym_lookup(constchar*name,
Fillintheawk_value_tstructurepointedtobyresultwiththevalueofthevariablenamedbythestringname,
whichisaregularCstring.wantedindicatesthetypeofvalueexpected.Returntrueiftheactualtypematches
wanted,andfalseotherwise.Inthelattercase,result>val_typeindicatestheactualtype(seeTable16.1).
awk_bool_tsym_update(constchar*name,awk_value_t*value);
Updatethevariablenamedbythestringname,whichisaregularCstring.Thevariableisaddedtogawkssymbol
tableifitisnotthere.Returntrueifeverythingworked,andfalseotherwise.
Changingtypes(scalartoarrayorviceversa)ofanexistingvariableisnotallowed,normaythisroutinebeused
toupdateanarray.Thisroutinecannotbeusedtoupdateanyofthepredefinedvariables(suchasARGCorNF).
Anextensioncanlookupthevalueofgawksspecialvariables.However,withtheexceptionofthePROCINFOarray,an
extensioncannotchangeanyofthosevariables.
CAUTION:ItispossibleforthelookupofPROCINFOtofail.Thishappensiftheawkprogrambeingrun
doesnotreferencePROCINFOinthiscase,gawkdoesntbothertocreatethearrayandpopulateit.
317/479
29/7/2016
TheGNUAwkUsersGuide
Next:Cachedvalues,Previous:Symboltablebyname,Up:SymbolTableAccess[Contents][Index]
16.4.10.2VariableAccessandUpdatebyCookie
Ascalarcookieisanopaquehandlethatprovidesaccesstoaglobalvariableorarray.Itisanoptimizationthatavoids
lookingupvariablesingawkssymboltableeverytimeaccessisneeded.Thiswasdiscussedearlier,inGeneralData
Types.
Thefollowingfunctionsletyouworkwithscalarcookies:
awk_bool_tsym_lookup_scalar(awk_scalar_tcookie,
Retrievethecurrentvalueofascalarcookie.Onceyouhaveobtainedascalarcookieusingsym_lookup(),youcan
usethisfunctiontogetitsvaluemoreefficiently.Returnfalseifthevaluecannotberetrieved.
awk_bool_tsym_update_scalar(awk_scalar_tcookie,awk_value_t*value);
Updatethevalueassociatedwithascalarcookie.ReturnfalseifthenewvalueisnotoftypeAWK_STRINGor
AWK_NUMBER.Heretoo,thepredefinedvariablesmaynotbeupdated.
Itisnotobviousatfirstglancehowtoworkwithscalarcookiesorwhattheirraisondtrereallyis.Intheory,the
sym_lookup()andsym_update()routinesareallyoureallyneedtoworkwithvariables.Forexample,youmighthave
codethatlooksupthevalueofavariable,evaluatesacondition,andthenpossiblychangesthevalueofthevariable
basedontheresultofthatevaluation,likeso:
/*do_magicdosomethingreallygreat*/
staticawk_value_t*
do_magic(intnargs,awk_value_t*result)
{
awk_value_tvalue;
if(sym_lookup("MAGIC_VAR",AWK_NUMBER,&value)
&&some_condition(value.num_value)){
value.num_value+=42;
sym_update("MAGIC_VAR",&value);
}
returnmake_number(0.0,result);
}
Thiscodelooks(andis)simpleandstraightforward.Sowhatstheproblem?
Well,considerwhathappensifawklevelcodeassociatedwithyourextensioncallsthemagic()function(implementedin
Cbydo_magic()),onceperrecord,whileprocessinghundredsofthousandsormillionsofrecords.TheMAGIC_VAR
variableislookedupinthesymboltableonceortwiceperfunctioncall!
Thesymboltablelookupisreallypureoverheaditisconsiderablymoreefficienttogetacookiethatrepresentsthe
variable,andusethattogetthevariablesvalueandupdateitasneeded.101
Thus,thewaytousecookiesisasfollows.First,installyourextensionsvariableingawkssymboltableusing
sym_update(),asusual.Thengetascalarcookieforthevariableusingsym_lookup():
staticawk_scalar_tmagic_var_cookie;/*cookieforMAGIC_VAR*/
staticvoid
my_extension_init()
{
awk_value_tvalue;
318/479
29/7/2016
TheGNUAwkUsersGuide
/*installinitialvalue*/
sym_update("MAGIC_VAR",make_number(42.0,&value));
/*getthecookie*/
sym_lookup("MAGIC_VAR",AWK_SCALAR,&value);
/*savethecookie*/
magic_var_cookie=value.scalar_cookie;
Next,usetheroutinesinthissectionforretrievingandupdatingthevaluethroughthecookie.Thus,do_magic()now
becomessomethinglikethis:
/*do_magicdosomethingreallygreat*/
staticawk_value_t*
{
awk_value_tvalue;
if(sym_lookup_scalar(magic_var_cookie,AWK_NUMBER,&value)
&&some_condition(value.num_value)){
value.num_value+=42;
sym_update_scalar(magic_var_cookie,&value);
}
returnmake_number(0.0,result);
}
NOTE:Thepreviouscodeomittederrorcheckingforpresentationpurposes.Yourextensioncodeshould
bemorerobustandcarefullycheckthereturnvaluesfromtheAPIfunctions.
Previous:Symboltablebycookie,Up:SymbolTableAccess[Contents][Index]
16.4.10.3CreatingandUsingCachedValues
Theroutinesinthissectionallowyoutocreateandreleasecachedvalues.Likescalarcookies,intheory,cachedvalues
arenotnecessary.YoucancreatenumbersandstringsusingthefunctionsinConstructorFunctions.Youcanthenassign
thosevaluestovariablesusingsym_update()orsym_update_scalar(),asyoulike.
However,youcanunderstandthepointofcachedvaluesifyourememberthateverystringvaluesstoragemustcome
fromgawk_malloc(),gawk_calloc(),orgawk_realloc().Ifyouhave20variables,allofwhichhavethesamestring
value,youmustcreate20identicalcopiesofthestring.102
Itisclearlymoreefficient,ifpossible,tocreateavalueonce,andthentellgawktoreusethevalueformultiplevariables.
Thatiswhattheroutinesinthissectionletyoudo.Thefunctionsareasfollows:
awk_bool_tcreate_value(awk_value_t*value,awk_value_cookie_t*result);
Createacachedstringornumericvaluefromvalueforefficientlaterassignment.OnlyvaluesoftypeAWK_NUMBER
andAWK_STRINGareallowed.Anyothertypeisrejected.AWK_UNDEFINEDcouldbeallowed,butdoingsowould
resultininferiorperformance.
awk_bool_trelease_value(awk_value_cookie_tvc);
Releasethememoryassociatedwithavaluecookieobtainedfromcreate_value().
Youusevaluecookiesinafashionsimilartothewayyouusescalarcookies.Intheextensioninitializationroutine,you
createthevaluecookie:
319/479
29/7/2016
TheGNUAwkUsersGuide
staticawk_value_cookie_tanswer_cookie;/*staticvaluecookie*/
staticvoid
my_extension_init()
{
awk_value_tvalue;
char*long_string;
size_tlong_string_len;
/*codefromearlier*/
/*fillinlong_stringandlong_string_len*/
make_malloced_string(long_string,long_string_len,&value);
create_value(&value,&answer_cookie);/*createcookie*/
Oncethevalueiscreated,youcanuseitasthevalueofanynumberofvariables:
staticawk_value_t*
{
awk_value_tnew_value;
/*asearlier*/
value.val_type=AWK_VALUE_COOKIE;
value.value_cookie=answer_cookie;
sym_update("VAR1",&value);
Usingvaluecookiesinthiswaysavesconsiderablestorage,asallofVAR1throughVAR100sharethesamevalue.
Youmightbewondering,Isthissharingproblematic?WhathappensifawkcodeassignsanewvaluetoVAR1areallthe
otherschangedtoo?
Thatsagreatquestion.Theansweristhatno,itsnotaproblem.Internally,gawkusesreferencecountedstrings.This
meansthatmanyvariablescansharethesamestringvalue,andgawkkeepstrackoftheusage.Whenavariablesvalue
changes,gawksimplydecrementsthereferencecountontheoldvalueandupdatesthevariabletousethenewvalue.
Finally,aspartofyourcleanupaction(seeExitCallbackFunctions)youshouldreleaseanycachedvaluesthatyou
created,usingrelease_value().
Next:ExtensionAPIVariables,Previous:SymbolTableAccess,Up:ExtensionAPIDescription[Contents][Index]
16.4.11ArrayManipulation
Theprimarydatastructure103inawkistheassociativearray(seeArrays).Extensionsneedtobeabletomanipulateawk
arrays.TheAPIprovidesanumberofdatastructuresforworkingwitharrays,functionsforworkingwithindividual
elements,andfunctionsforworkingwitharraysasawhole.Thisincludestheabilitytoflattenanarraysothatitis
easyforCcodetotraverseeveryelementinanarray.Thearraydatastructuresintegratenicelywiththedatastructures
forvaluestomakeiteasytobothworkwithandcreatetruearraysofarrays(seeGeneralDataTypes).
ArrayDataTypes: Datatypesforworkingwitharrays.
ArrayFunctions: Functionsforworkingwitharrays.
FlatteningArrays: Howtoflattenarrays.
CreatingArrays: Howtocreateandpopulatearrays.
320/479
29/7/2016
TheGNUAwkUsersGuide
Next:ArrayFunctions,Up:ArrayManipulation[Contents][Index]
16.4.11.1ArrayDataTypes
Thedatatypesassociatedwitharraysareasfollows:
typedefvoid*awk_array_t;
Ifyourequestthevalueofanarrayvariable,yougetbackanawk_array_tvalue.Thisvalueisopaque104tothe
extensionituniquelyidentifiesthearraybutcanonlybeusedbypassingitintoAPIfunctionsorreceivingit
fromAPIfunctions.ThisisverysimilartowayFILE*valuesareusedwiththe<stdio.h>libraryroutines.
typedefstructawk_element{
/*conveniencelinkedlistpointer,notusedbygawk*/
structawk_element*next;
enum{
AWK_ELEMENT_DEFAULT=0,/*setbygawk*/
AWK_ELEMENT_DELETE=1/*setbyextension*/
}flags;
awk_value_tindex;
awk_value_tvalue;
}awk_element_t;
Theawk_element_tisaflattenedarrayelement.awkproducesanarrayoftheseinsidetheawk_flat_array_t(see
thenextitem).Individualelementsmaybemarkedfordeletion.Newelementsmustbeaddedindividually,oneat
atime,usingtheseparateAPIforthatpurpose.Thefieldsareasfollows:
structawk_element*next;
Thispointerisfortheconvenienceofextensionwriters.Itallowsanextensiontocreatealinkedlistofnew
elementsthatcanthenbeaddedtoanarrayinaloopthattraversesthelist.
enum{}flags;
Asetofflagvaluesthatconveyinformationbetweentheextensionandgawk.Currentlythereisonlyone:
AWK_ELEMENT_DELETE.Settingitcausesgawktodeletetheelementfromtheoriginalarrayuponreleaseofthe
flattenedarray.
index
value
Theindexandvalueoftheelement,respectively.Allmemorypointedtobyindexandvaluebelongsto
gawk.
typedefstructawk_flat_array{
awk_constvoid*awk_constopaque1;/*forusebygawk*/
awk_constvoid*awk_constopaque2;/*forusebygawk*/
awk_constsize_tcount;/*howmanyelements*/
awk_element_telements[1];/*willbeextended*/
}awk_flat_array_t;
Thisisaflattenedarray.Whenanextensiongetsoneofthesefromgawk,theelementsarrayisofactualsizecount.
Theopaque1andopaque2pointersareforusebygawkthereforetheyaremarkedawk_constsothattheextension
cannotmodifythem.
Next:FlatteningArrays,Previous:ArrayDataTypes,Up:ArrayManipulation[Contents][Index]
16.4.11.2ArrayFunctions
Thefollowingfunctionsrelatetoindividualarrayelements:
321/479
29/7/2016
TheGNUAwkUsersGuide
awk_bool_tget_element_count(awk_array_ta_cookie,size_t*count);
Forthearrayrepresentedbya_cookie,placein*countthenumberofelementsitcontains.Asubarraycountsasa
singleelement.Returnfalseifthereisanerror.
awk_bool_tget_array_element(awk_array_ta_cookie,
constawk_value_t*constindex,
Forthearrayrepresentedbya_cookie,returnin*resultthevalueoftheelementwhoseindexisindex.wanted
specifiesthetypeofvalueyouwishtoretrieve.Returnfalseifwanteddoesnotmatchtheactualtypeorifindexis
notinthearray(seeTable16.1).
Thevalueforindexcanbenumeric,inwhichcasegawkconvertsittoastring.Usingnonintegralvaluesis
possible,butrequiresthatyouunderstandhowsuchvaluesareconvertedtostrings(seeConversion)thus,using
integralvaluesissafest.
Aswithallstringspassedintogawkfromanextension,thestringvalueofindexmustcomefromgawk_malloc(),
gawk_calloc(),orgawk_realloc(),andgawkreleasesthestorage.
awk_bool_tset_array_element(awk_array_ta_cookie,
constawk_value_t*constindex,
constawk_value_t*constvalue);
Inthearrayrepresentedbya_cookie,createormodifytheelementwhoseindexisgivenbyindex.TheARGVand
ENVIRONarraysmaynotbechanged,althoughthePROCINFOarraycanbe.
awk_bool_tset_array_element_by_elem(awk_array_ta_cookie,
awk_element_telement);
Likeset_array_element(),buttaketheindexandvaluefromelement.Thisisaconveniencemacro.
awk_bool_tdel_array_element(awk_array_ta_cookie,
constawk_value_t*constindex);
Removetheelementwiththegivenindexfromthearrayrepresentedbya_cookie.Returntrueiftheelementwas
removed,orfalseiftheelementdidnotexistinthearray.
Thefollowingfunctionsrelatetoarraysasawhole:
awk_array_tcreate_array(void);
Createanewarraytowhichelementsmaybeadded.SeeCreatingArrays,foradiscussionofhowtocreateanew
arrayandaddelementstoit.
awk_bool_tclear_array(awk_array_ta_cookie);
Clearthearrayrepresentedbya_cookie.Returnfalseiftherewassomekindofproblem,trueotherwise.Thearray
remainsanarray,butaftercallingthisfunction,ithasnoelements.Thisisequivalenttousingthedelete
statement(seeDelete).
awk_bool_tflatten_array(awk_array_ta_cookie,awk_flat_array_t**data);
Forthearrayrepresentedbya_cookie,createanawk_flat_array_tstructureandfillitin.Setthepointerwhose
addressispassedasdatatopointtothisstructure.Returntrueuponsuccess,orfalseotherwise.SeeFlattening
Arrays,foradiscussionofhowtoflattenanarrayandworkwithit.
awk_bool_trelease_flattened_array(awk_array_ta_cookie,
awk_flat_array_t*data);
Whendonewithaflattenedarray,releasethestorageusingthisfunction.Youmustpassinboththeoriginalarray
cookieandtheaddressofthecreatedawk_flat_array_tstructure.Thefunctionreturnstrueuponsuccess,false
322/479
29/7/2016
TheGNUAwkUsersGuide
otherwise.
Next:CreatingArrays,Previous:ArrayFunctions,Up:ArrayManipulation[Contents][Index]
16.4.11.3WorkingWithAllTheElementsofanArray
ToflattenanarrayistocreateastructurethatrepresentsthefullarrayinafashionthatmakesiteasyforCcodeto
traversetheentirearray.Someofthecodeinextension/testext.cdoesthis,andalsoservesasaniceexampleshowing
howtousetheAPIs.
Wewalkthroughthatpartofthecodeonestepatatime.First,thegawkscriptthatdrivesthetestextension:
@load"testext"
BEGIN{
n=split("blackyrustysophieraincloudlucky",pets)
printf("petshas%delements\n",length(pets))
ret=dump_array_and_delete("pets","3")
printf("dump_array_and_delete(pets)returned%d\n",ret)
if("3"inpets)
printf("dump_array_and_delete()didNOTremoveindex\"3\"!\n")
else
printf("dump_array_and_delete()didremoveindex\"3\"!\n")
print""
}
Thiscodecreatesanarraywithsplit()(seeStringFunctions)andthencallsdump_array_and_delete().Thatfunction
looksupthearraywhosenameispassedasthefirstargument,anddeletestheelementattheindexpassedinthesecond
argument.Theawkcodethenprintsthereturnvalueandchecksiftheelementwasindeeddeleted.HereistheCcode
thatimplementsdump_array_and_delete().Ithasbeeneditedslightlyforpresentation.
Thefirstpartdeclaresvariables,setsupthedefaultreturnvalueinresult,andchecksthatthefunctionwascalledwith
thecorrectnumberofarguments:
staticawk_value_t*
dump_array_and_delete(intnargs,awk_value_t*result)
{
awk_value_tvalue,value2,value3;
awk_flat_array_t*flat_array;
size_tcount;
char*name;
inti;
assert(result!=NULL);
make_number(0.0,result);
if(nargs!=2){
printf("dump_array_and_delete:nargsnotright"
"(%dshouldbe2)\n",nargs);
gotoout;
}
Thefunctionthenproceedsinsteps,asfollows.First,retrievethenameofthearray,passedasthefirstargument,
followedbythearrayitself.Ifeitheroperationfails,printanerrormessageandreturn:
/*getargumentnamedarrayasflatarrayandprintit*/
if(get_argument(0,AWK_STRING,&value)){
name=value.str_value.str;
if(sym_lookup(name,AWK_ARRAY,&value2))
printf("dump_array_and_delete:sym_lookupof%spassed\n",
name);
else{
printf("dump_array_and_delete:sym_lookupof%sfailed\n",
name);
323/479
29/7/2016
TheGNUAwkUsersGuide
gotoout;
}
}else{
printf("dump_array_and_delete:get_argument(0)failed\n");
gotoout;
}
FortestingpurposesandtomakesurethattheCcodeseesthesamenumberofelementsastheawkcode,thesecondstep
istogetthecountofelementsinthearrayandprintit:
if(!get_element_count(value2.array_cookie,&count)){
printf("dump_array_and_delete:get_element_countfailed\n");
gotoout;
}
printf("dump_array_and_delete:incomingsizeis%lu\n",
(unsignedlong)count);
Thethirdstepistoactuallyflattenthearray,andthentodoublecheckthatthecountintheawk_flat_array_tisthe
sameasthecountjustretrieved:
if(!flatten_array(value2.array_cookie,&flat_array)){
printf("dump_array_and_delete:couldnotflattenarray\n");
gotoout;
}
if(flat_array>count!=count){
printf("dump_array_and_delete:flat_array>count(%lu)"
"!=count(%lu)\n",
(unsignedlong)flat_array>count,
(unsignedlong)count);
gotoout;
}
Thefourthstepistoretrievetheindexoftheelementtobedeleted,whichwaspassedasthesecondargument.
Rememberthatargumentcountspassedtoget_argument()arezerobased,andthusthesecondargumentisnumbered
one:
if(!get_argument(1,AWK_STRING,&value3)){
printf("dump_array_and_delete:get_argument(1)failed\n");
gotoout;
}
Thefifthstepiswheretherealworkisdone.Thefunctionloopsovereveryelementinthearray,printingtheindex
andelementvalues.Inaddition,uponfindingtheelementwiththeindexthatissupposedtobedeleted,thefunctionsets
theAWK_ELEMENT_DELETEbitintheflagsfieldoftheelement.Whenthearrayisreleased,gawktraversestheflattened
array,anddeletesanyelementsthathavethisflagbitset:
for(i=0;i<flat_array>count;i++){
printf("\t%s[\"%.*s\"]=%s\n",
name,
(int)flat_array>elements[i].index.str_value.len,
flat_array>elements[i].index.str_value.str,
valrep2str(&flat_array>elements[i].value));
if(strcmp(value3.str_value.str,
flat_array>elements[i].index.str_value.str)==0){
flat_array>elements[i].flags|=AWK_ELEMENT_DELETE;
printf("dump_array_and_delete:markingelement\"%s\""
"fordeletion\n",
flat_array>elements[i].index.str_value.str);
}
}
Thesixthstepistoreleasetheflattenedarray.Thistellsgawkthattheextensionisnolongerusingthearray,andthatit
shoulddeleteanyelementsmarkedfordeletion.gawkalsofreesanystoragethatwasallocated,soyoushouldnotusethe
324/479
29/7/2016
TheGNUAwkUsersGuide
pointer(flat_arrayinthiscode)onceyouhavecalledrelease_flattened_array():
if(!release_flattened_array(value2.array_cookie,flat_array)){
printf("dump_array_and_delete:couldnotreleaseflattenedarray\n");
gotoout;
}
Finally,becauseeverythingwassuccessful,thefunctionsetsthereturnvaluetosuccess,andreturns:
make_number(1.0,result);
out:
returnresult;
}
Hereistheoutputfromrunningthispartofthetest:
petshas5elements
dump_array_and_delete:sym_lookupofpetspassed
dump_array_and_delete:incomingsizeis5
pets["1"]="blacky"
pets["2"]="rusty"
pets["3"]="sophie"
dump_array_and_delete:markingelement"3"fordeletion
pets["4"]="raincloud"
pets["5"]="lucky"
dump_array_and_delete(pets)returned1
dump_array_and_delete()didremoveindex"3"!
Previous:FlatteningArrays,Up:ArrayManipulation[Contents][Index]
16.4.11.4HowToCreateandPopulateArrays
Besidesworkingwitharrayscreatedbyawkcode,youcancreatearraysandpopulatethemasyouseefit,andthenawk
codecanaccessthemandmanipulatethem.
Therearetwoimportantpointsaboutcreatingarraysfromextensioncode:
Youmustinstallanewarrayintogawkssymboltableimmediatelyuponcreatingit.Onceyouhavedoneso,you
canthenpopulatethearray.
Similarly,ifinstallinganewarrayasasubarrayofanexistingarray,youmustaddthenewarraytoitsparent
beforeaddinganyelementstoit.
Thus,thecorrectwaytobuildanarrayistoworktopdown.Createthearray,andimmediatelyinstallitin
gawkssymboltableusingsym_update(),orinstallitasanelementinapreviouslyexistingarrayusing
set_array_element().Weshowexamplecodeshortly.
Duetogawkinternals,afterusingsym_update()toinstallanarrayintogawk,youhavetoretrievethearraycookie
fromthevaluepassedintosym_update()beforedoinganythingelsewithit,likeso:
awk_value_tvalue;
awk_array_tnew_array;
new_array=create_array();
val.val_type=AWK_ARRAY;
val.array_cookie=new_array;
/*installarrayinthesymboltable*/
sym_update("array",&val);
new_array=val.array_cookie;/*YOUMUSTDOTHIS*/
Ifinstallinganarrayasasubarray,youmustalsoretrievethevalueofthearraycookieafterthecallto
325/479
29/7/2016
TheGNUAwkUsersGuide
set_element().
ThefollowingCcodeisasimpletestextensiontocreateanarraywithtworegularelementsandwithasubarray.The
leading#includedirectivesandboilerplatevariabledeclarations(seeExtensionAPIBoilerplate)areomittedforbrevity.
Thefirststepistocreateanewarrayandtheninstallitinthesymboltable:
/*create_new_arraycreateanamedarray*/
staticvoid
create_new_array()
{
awk_array_ta_cookie;
awk_array_tsubarray;
awk_value_tindex,value;
a_cookie=create_array();
value.val_type=AWK_ARRAY;
value.array_cookie=a_cookie;
if(!sym_update("new_array",&value))
printf("create_new_array:sym_update(\"new_array\")failed!\n");
a_cookie=value.array_cookie;
Notehowa_cookieisresetfromthearray_cookiefieldinthevaluestructure.
Thesecondstepistoinstalltworegularvaluesintonew_array:
(void)make_const_string("hello",5,&index);
(void)make_const_string("world",5,&value);
if(!set_array_element(a_cookie,&index,&value)){
printf("fill_in_array:set_array_elementfailed\n");
return;
}
(void)make_const_string("answer",6,&index);
(void)make_number(42.0,&value);
return;
}
Thethirdstepistocreatethesubarrayandinstallit:
(void)make_const_string("subarray",8,&index);
subarray=create_array();
value.val_type=AWK_ARRAY;
value.array_cookie=subarray;
return;
}
subarray=value.array_cookie;
Thefinalstepistopopulatethesubarraywithitsownelement:
(void)make_const_string("foo",3,&index);
(void)make_const_string("bar",3,&value);
if(!set_array_element(subarray,&index,&value)){
return;
}
}
Hereisasamplescriptthatloadstheextensionandthendumpsthearray:
@load"subarray"
326/479
29/7/2016
TheGNUAwkUsersGuide
functiondumparray(name,array,i)
{
for(iinarray)
if(isarray(array[i]))
dumparray(name"[\""i"\"]",array[i])
else
printf("%s[\"%s\"]=%s\n",name,i,array[i])
}
BEGIN{
dumparray("new_array",new_array);
}
Hereistheresultofrunningthescript:
$AWKLIBPATH=$PWD./gawkfsubarray.awk
|new_array["subarray"]["foo"]=bar
|new_array["hello"]=world
|new_array["answer"]=42
(SeeFindingExtensions,formoreinformationontheAWKLIBPATHenvironmentvariable.)
Next:ExtensionAPIBoilerplate,Previous:ArrayManipulation,Up:ExtensionAPIDescription[Contents][Index]
16.4.12APIVariables
TheAPIprovidestwosetsofvariables.ThefirstprovidesinformationabouttheversionoftheAPI(bothwithwhichthe
extensionwascompiled,andwithwhichgawkwascompiled).Thesecondprovidesinformationabouthowgawkwas
invoked.
ExtensionVersioning:
APIVersioninformation.
ExtensionAPIInformationalVariables: Variablesprovidinginformationaboutgawksinvocation.
Next:ExtensionAPIInformationalVariables,Up:ExtensionAPIVariables[Contents][Index]
16.4.12.1APIVersionConstantsandVariables
TheAPIprovidesbothamajorandaminorversionnumber.TheAPIversionsareavailableatcompiletimeas
constants:
GAWK_API_MAJOR_VERSION
ThemajorversionoftheAPI
GAWK_API_MINOR_VERSION
TheminorversionoftheAPI
TheminorversionincreaseswhennewfunctionsareaddedtotheAPI.Suchnewfunctionsarealwaysaddedtotheend
oftheAPIstruct.
Themajorversionincreases(andtheminorversionisresettozero)ifanyofthedatatypeschangesizeormember
order,orifanyoftheexistingfunctionschangesignature.
ItcouldhappenthatanextensionmaybecompiledagainstoneversionoftheAPIbutloadedbyaversionofgawkusing
adifferentversion.Forthisreason,themajorandminorAPIversionsoftherunninggawkareincludedintheAPIstruct
asreadonlyconstantintegers:
api>major_version
327/479
29/7/2016
TheGNUAwkUsersGuide
Themajorversionoftherunninggawk
api>minor_version
Theminorversionoftherunninggawk
ItisuptotheextensiontodecideifthereareAPIincompatibilities.Typically,achecklikethisisenough:
if(api>major_version!=GAWK_API_MAJOR_VERSION
||api>minor_version<GAWK_API_MINOR_VERSION){
fprintf(stderr,"foo_extension:versionmismatchwithgawk!\n");
fprintf(stderr,"\tmyversion(%d,%d),gawkversion(%d,%d)\n",
GAWK_API_MAJOR_VERSION,GAWK_API_MINOR_VERSION,
api>major_version,api>minor_version);
exit(1);
}
Suchcodeisincludedintheboilerplatedl_load_func()macroprovidedingawkapi.h(discussedinExtensionAPI
Boilerplate).
Previous:ExtensionVersioning,Up:ExtensionAPIVariables[Contents][Index]
16.4.12.2InformationalVariables
TheAPIprovidesaccesstoseveralvariablesthatdescribewhetherthecorrespondingcommandlineoptionswere
enabledwhengawkwasinvoked.Thevariablesare:
do_debug
Thisvariableistrueifgawkwasinvokedwithdebugoption.
do_lint
Thisvariableistrueifgawkwasinvokedwithlintoption.
do_mpfr
Thisvariableistrueifgawkwasinvokedwithbignumoption.
do_profile
Thisvariableistrueifgawkwasinvokedwithprofileoption.
do_sandbox
Thisvariableistrueifgawkwasinvokedwithsandboxoption.
do_traditional
Thisvariableistrueifgawkwasinvokedwithtraditionaloption.
Thevalueofdo_lintcanchangeifawkcodemodifiestheLINTpredefinedvariable(seeBuiltinVariables).Theothers
shouldnotchangeduringexecution.
Previous:ExtensionAPIVariables,Up:ExtensionAPIDescription[Contents][Index]
16.4.13BoilerplateCode
Asmentionedearlier(seeExtensionMechanismOutline),thefunctiondefinitionsaspresentedarereallymacros.Touse
328/479
29/7/2016
TheGNUAwkUsersGuide
thesemacros,yourextensionmustprovideasmallamountofboilerplatecode(variablesandfunctions)towardthetop
ofyoursourcefile,usingpredefinednamesasdescribedhere.Theboilerplateneededisalsoprovidedincommentsin
thegawkapi.hheaderfile:
/*Boilerplatecode:*/
staticgawk_api_t*constapi;
staticawk_ext_id_text_id;
staticconstchar*ext_version=NULL;/*or="somestring"*/
staticawk_ext_func_tfunc_table[]={
{"name",do_name,1},
/**/
};
/*EITHER:*/
staticawk_bool_t(*init_func)(void)=NULL;
/*OR:*/
staticawk_bool_t
init_my_extension(void)
{
}
staticawk_bool_t(*init_func)(void)=init_my_extension;
dl_load_func(func_table,some_name,"name_space_in_quotes")
Thesevariablesandfunctionsareasfollows:
ThisassertsthattheextensioniscompatiblewiththeGNUGPL(seeCopying).Ifyourextensiondoesnothave
this,gawkwillnotloadit(seePluginLicense).
staticgawk_api_t*constapi;
Thisglobalstaticvariableshouldbesettopointtothegawk_api_tpointerthatgawkpassestoyourdl_load()
function.Thisvariableisusedbyallofthemacros.
staticawk_ext_id_text_id;
Thisglobalstaticvariableshouldbesettotheawk_ext_id_tvaluethatgawkpassestoyourdl_load()function.
Thisvariableisusedbyallofthemacros.
staticconstchar*ext_version=NULL;/*or="somestring"*/
ThisglobalstaticvariableshouldbeseteithertoNULL,ortopointtoastringgivingthenameandversionofyour
extension.
staticawk_ext_func_tfunc_table[]={};
Thisisanarrayofoneormoreawk_ext_func_tstructures,asdescribedearlier(seeExtensionFunctions).Itcan
thenbeloopedoverformultiplecallstoadd_ext_func().
staticawk_bool_t(*init_func)(void)=NULL;
OR
staticawk_bool_tinit_my_extension(void){}
staticawk_bool_t(*init_func)(void)=init_my_extension;
Ifyouneedtodosomeinitializationwork,youshoulddefineafunctionthatdoesit(createsvariables,opensfiles,
329/479
29/7/2016
TheGNUAwkUsersGuide
etc.)andthendefinetheinit_funcpointertopointtoyourfunction.Thefunctionshouldreturnawk_falseupon
failure,orawk_trueifeverythinggoeswell.
Ifyoudontneedtodoanyinitialization,definethepointerandinitializeittoNULL.
dl_load_func(func_table,some_name,"name_space_in_quotes")
Thismacroexpandstoadl_load()functionthatperformsallthenecessaryinitializations.
Thepointofallthevariablesandarraysistoletthedl_load()function(fromthedl_load_func()macro)doallthe
standardwork.Itdoesthefollowing:
1.ChecktheAPIversions.Iftheextensionmajorversiondoesnotmatchgawks,oriftheextensionminorversionis
greaterthangawks,itprintsafatalerrormessageandexits.
2.Loadthefunctionsdefinedinfunc_table.Ifanyofthemfailstoload,itprintsawarningmessagebutcontinues
on.
3.Iftheinit_funcpointerisnotNULL,callthefunctionitpointsto.Ifitreturnsawk_false,printawarningmessage.
4.Ifext_versionisnotNULL,registertheversionstringwithgawk.
Next:ExtensionExample,Previous:ExtensionAPIDescription,Up:DynamicExtensions[Contents][Index]
16.5HowgawkFindsExtensions
Compiledextensionshavetobeinstalledinadirectorywheregawkcanfindthem.Ifgawkisconfiguredandbuiltinthe
defaultfashion,thedirectoryinwhichtofindextensionsis/usr/local/lib/gawk.Youcanalsospecifyasearchpath
withalistofdirectoriestosearchforcompiledextensions.SeeAWKLIBPATHVariable,formoreinformation.
Next:ExtensionSamples,Previous:FindingExtensions,Up:DynamicExtensions[Contents][Index]
16.6Example:SomeFileFunctions
Nomatterwhereyougo,thereyouare.
BuckarooBanzai
Twousefulfunctionsthatarenotinawkarechdir()(sothatanawkprogramcanchangeitsdirectory)andstat()(sothat
anawkprogramcangatherinformationaboutafile).InordertoillustratetheAPIinaction,thissectionimplements
thesefunctionsforgawkinanextension.
InternalFileDescription: Whatthenewfunctionswilldo.
InternalFileOps:
Thecodeforinternalfileoperations.
UsingInternalFileOps: Howtouseanexternalextension.
Next:InternalFileOps,Up:ExtensionExample[Contents][Index]
16.6.1Usingchdir()andstat()
Thissectionshowshowtousethenewfunctionsattheawkleveloncetheyvebeenintegratedintotherunninggawk
interpreter.Usingchdir()isverystraightforward.Ittakesoneargument,thenewdirectorytochangeto:
@load"filefuncs"
newdir="/home/arnold/funstuff"
ret=chdir(newdir)
if(ret<0){
printf("couldnotchangeto%s:%s\n",newdir,ERRNO)>"/dev/stderr"
330/479
29/7/2016
TheGNUAwkUsersGuide
exit1
}
Thereturnvalueisnegativeifthechdir()failed,andERRNO(seeBuiltinVariables)issettoastringindicatingtheerror.
Usingstat()isabitmorecomplicated.TheCstat()functionfillsinastructurethathasafairamountofinformation.
Therightwaytomodelthisinawkistofillinanassociativearraywiththeappropriateinformation:
file="/home/arnold/.profile"
ret=stat(file,fdata)
if(ret<0){
printf("couldnotstat%s:%s\n",
file,ERRNO)>"/dev/stderr"
exit1
}
printf("sizeof%sis%dbytes\n",file,fdata["size"])
Thestat()functionalwaysclearsthedataarray,evenifthestat()fails.Itfillsinthefollowingelements:
"name"
Thenameofthefilethatwasstat()ed.
"dev"
"ino"
Thefilesdeviceandinodenumbers,respectively.
"mode"
Thefilesmode,asanumericvalue.Thisincludesboththefilestypeanditspermissions.
"nlink"
Thenumberofhardlinks(directoryentries)thefilehas.
"uid"
"gid"
ThenumericuserandgroupIDnumbersofthefilesowner.
"size"
Thesizeinbytesofthefile.
"blocks"
Thenumberofdiskblocksthefileactuallyoccupies.Thismaynotbeafunctionofthefilessizeifthefilehas
holes.
"atime"
"mtime"
"ctime"
Thefileslastaccess,modification,andinodeupdatetimes,respectively.Thesearenumerictimestamps,suitable
forformattingwithstrftime()(seeTimeFunctions).
"pmode"
Thefilesprintablemode.Thisisastringrepresentationofthefilestypeandpermissions,suchasisproduced
bylslforexample,"drwxrxrx".
"type"
331/479
29/7/2016
TheGNUAwkUsersGuide
Aprintablestringrepresentationofthefilestype.Thevalueisoneofthefollowing:
"blockdev"
"chardev"
Thefileisablockorcharacterdevice(specialfile).
"directory"
Thefileisadirectory.
"fifo"
Thefileisanamedpipe(alsoknownasaFIFO).
"file"
Thefileisjustaregularfile.
"socket"
ThefileisanAF_UNIX(Unixdomain)socketinthefilesystem.
"symlink"
Thefileisasymboliclink.
"devbsize"
Thesizeofablockfortheelementindexedby"blocks".ThisinformationisderivedfromeithertheDEV_BSIZE
constantdefinedin<sys/param.h>onmostsystems,ortheS_BLKSIZEconstantin<sys/stat.h>onBSDsystems.
Forsomeothersystems,aprioriknowledgeisusedtoprovideavalue.Wherenovaluecanbedetermined,it
defaultsto512.
Severaladditionalelementsmaybepresent,dependingupontheoperatingsystemandthetypeofthefile.Youcantest
fortheminyourawkprogrambyusingtheinoperator(seeReferencetoElements):
"blksize"
ThepreferredblocksizeforI/Otothefile.ThisfieldisnotpresentonallPOSIXlikesystemsintheCstat
structure.
"linkval"
Ifthefileisasymboliclink,thiselementisthenameofthefilethelinkpointsto(i.e.,thevalueofthelink).
"rdev"
"major"
"minor"
Ifthefileisablockorcharacterdevicefile,thenthesevaluesrepresentthenumericdevicenumberandthemajor
andminorcomponentsofthatnumber,respectively.
Next:UsingInternalFileOps,Previous:InternalFileDescription,Up:ExtensionExample[Contents][Index]
16.6.2CCodeforchdir()andstat()
HereistheCcodefortheseextensions.105
Thefileincludesanumberofstandardheaderfiles,andthenincludesthegawkapi.hheaderfile,whichprovidestheAPI
definitions.ThosearefollowedbythenecessaryvariabledeclarationstomakeuseoftheAPImacrosandboilerplate
332/479
29/7/2016
TheGNUAwkUsersGuide
code(seeExtensionAPIBoilerplate):
#ifdefHAVE_CONFIG_H
#include<config.h>
#endif
#include<stdio.h>
#include<assert.h>
#include<errno.h>
#include<stdlib.h>
#include<string.h>
#include<unistd.h>
#include<sys/types.h>
#include<sys/stat.h>
#include"gawkapi.h"
#include"gettext.h"
#define_(msgid)gettext(msgid)
#defineN_(msgid)msgid
#include"gawkfts.h"
#include"stack.h"
staticconstgawk_api_t*api;/*forconveniencemacrostowork*/
staticawk_ext_id_t*ext_id;
staticawk_bool_tinit_filefuncs(void);
staticawk_bool_t(*init_func)(void)=init_filefuncs;
staticconstchar*ext_version="filefuncsextension:version1.0";
Byconvention,foranawkfunctionfoo(),theCfunctionthatimplementsitiscalleddo_foo().Thefunctionshouldhave
twoarguments.Thefirstisanint,usuallycallednargs,thatrepresentsthenumberofactualargumentsforthefunction.
Thesecondisapointertoanawk_value_tstructure,usuallynamedresult:
/*do_chdirprovidedynamicallyloadedchdir()functionforgawk*/
staticawk_value_t*
do_chdir(intnargs,awk_value_t*result)
{
awk_value_tnewdir;
intret=1;
if(do_lint&&nargs!=1)
lintwarn(ext_id,
_("chdir:calledwithincorrectnumberofarguments,"
"expecting1"));
Thenewdirvariablerepresentsthenewdirectorytochangeto,whichisretrievedwithget_argument().Notethatthe
firstargumentisnumberedzero.
Iftheargumentisretrievedsuccessfully,thefunctioncallsthechdir()systemcall.Ifthechdir()fails,ERRNOisupdated:
if(get_argument(0,AWK_STRING,&newdir)){
ret=chdir(newdir.str_value.str);
if(ret<0)
update_ERRNO_int(errno);
}
Finally,thefunctionreturnsthereturnvaluetotheawklevel:
returnmake_number(ret,result);
}
333/479
29/7/2016
TheGNUAwkUsersGuide
Thestat()extensionismoreinvolved.Firstcomesafunctionthatturnsanumericmodeintoaprintablerepresentation
(e.g.,octal0644becomesrwrr).Thisisomittedhereforbrevity:
/*format_modeturnastatmodefieldintosomethingreadable*/
staticchar*
format_mode(unsignedlongfmode)
{
Nextcomesafunctionforreadingsymboliclinks,whichisalsoomittedhereforbrevity:
/*read_symlinkreadasymboliclinkintoanallocatedbuffer.
*/
staticchar*
read_symlink(constchar*fname,size_tbufsize,ssize_t*linksize)
{
Twohelperfunctionssimplifyenteringvaluesinthearraythatwillcontaintheresultofthestat():
/*array_setsetanarrayelement*/
staticvoid
array_set(awk_array_tarray,constchar*sub,awk_value_t*value)
{
awk_value_tindex;
set_array_element(array,
make_const_string(sub,strlen(sub),&index),
value);
}
/*array_set_numericsetanarrayelementwithanumber*/
staticvoid
array_set_numeric(awk_array_tarray,constchar*sub,doublenum)
{
awk_value_ttmp;
array_set(array,sub,make_number(num,&tmp));
}
Thefollowingfunctiondoesmostoftheworktofillintheawk_array_tresultarraywithvaluesobtainedfromavalid
structstat.Thisworkisdoneinaseparatefunctiontosupportthestat()functionforgawkandalsotosupportthe
fts()extension,whichisincludedinthesamefilebutwhosecodeisnotshownhere(seeExtensionSampleFile
Functions).
Thefirstpartofthefunctionisvariabledeclarations,includingatabletomapfiletypestostrings:
/*fill_stat_arraydotheworktofillanarraywithstatinfo*/
staticint
fill_stat_array(constchar*name,awk_array_tarray,structstat*sbuf)
{
char*pmode;/*printablemode*/
constchar*type="unknown";
awk_value_ttmp;
staticstructftype_map{
unsignedintmask;
constchar*type;
}ftype_map[]={
{S_IFREG,"file"},
334/479
29/7/2016
TheGNUAwkUsersGuide
{S_IFBLK,"blockdev"},
{S_IFCHR,"chardev"},
{S_IFDIR,"directory"},
#ifdefS_IFSOCK
{S_IFSOCK,"socket"},
#endif
#ifdefS_IFIFO
{S_IFIFO,"fifo"},
#endif
#ifdefS_IFLNK
{S_IFLNK,"symlink"},
#endif
#ifdefS_IFDOOR/*Solarisweirdness*/
{S_IFDOOR,"door"},
#endif/*S_IFDOOR*/
};
intj,k;
Thedestinationarrayiscleared,andthencodefillsinvariouselementsbasedonvaluesinthestructstat:
/*emptyoutthearray*/
clear_array(array);
/*fillinthearray*/
array_set(array,"name",make_const_string(name,strlen(name),
&tmp));
array_set_numeric(array,"dev",sbuf>st_dev);
array_set_numeric(array,"ino",sbuf>st_ino);
array_set_numeric(array,"mode",sbuf>st_mode);
array_set_numeric(array,"nlink",sbuf>st_nlink);
array_set_numeric(array,"uid",sbuf>st_uid);
array_set_numeric(array,"gid",sbuf>st_gid);
array_set_numeric(array,"size",sbuf>st_size);
array_set_numeric(array,"blocks",sbuf>st_blocks);
array_set_numeric(array,"atime",sbuf>st_atime);
array_set_numeric(array,"mtime",sbuf>st_mtime);
array_set_numeric(array,"ctime",sbuf>st_ctime);
/*forblockandcharacterdevices,addrdev,
majorandminornumbers*/
if(S_ISBLK(sbuf>st_mode)||S_ISCHR(sbuf>st_mode)){
array_set_numeric(array,"rdev",sbuf>st_rdev);
array_set_numeric(array,"major",major(sbuf>st_rdev));
array_set_numeric(array,"minor",minor(sbuf>st_rdev));
}
Thelatterpartofthefunctionmakesselectiveadditionstothedestinationarray,dependingupontheavailabilityof
certainmembersand/orthetypeofthefile.Itthenreturnszero,forsuccess:
#ifdefHAVE_STRUCT_STAT_ST_BLKSIZE
array_set_numeric(array,"blksize",sbuf>st_blksize);
#endif/*HAVE_STRUCT_STAT_ST_BLKSIZE*/
pmode=format_mode(sbuf>st_mode);
array_set(array,"pmode",make_const_string(pmode,strlen(pmode),
&tmp));
/*forsymboliclinks,addalinkvalfield*/
if(S_ISLNK(sbuf>st_mode)){
char*buf;
ssize_tlinksize;
if((buf=read_symlink(name,sbuf>st_size,
&linksize))!=NULL)
array_set(array,"linkval",
make_malloced_string(buf,linksize,&tmp));
else
warning(ext_id,_("stat:unabletoreadsymboliclink`%s'"),
335/479
29/7/2016
TheGNUAwkUsersGuide
name);
}
/*addatypefield*/
type="unknown";/*shouldn'thappen*/
for(j=0,k=sizeof(ftype_map)/sizeof(ftype_map[0]);j<k;j++){
if((sbuf>st_mode&S_IFMT)==ftype_map[j].mask){
type=ftype_map[j].type;
break;
}
}
array_set(array,"type",make_const_string(type,strlen(type),&tmp));
return0;
}
Thethirdargumenttostat()wasnotdiscussedpreviously.Thisargumentisoptional.Ifpresent,itcausesdo_stat()to
usethestat()systemcallinsteadofthelstat()systemcall.Thisisdonebyusingafunctionpointer:statfunc.
statfuncisinitializedtopointtolstat()(insteadofstat())togetthefileinformation,incasethefileisasymbolic
link.However,ifthethirdargumentisincluded,statfuncissettopointtostat(),instead.
Hereisthedo_stat()function,whichstartswithvariabledeclarationsandargumentchecking:
/*do_statprovideastat()functionforgawk*/
staticawk_value_t*
do_stat(intnargs,awk_value_t*result)
{
awk_value_tfile_param,array_param;
char*name;
awk_array_tarray;
intret;
structstatsbuf;
/*defaultislstat()*/
int(*statfunc)(constchar*path,structstat*sbuf)=lstat;
if(nargs!=2&&nargs!=3){
if(do_lint)
lintwarn(ext_id,
_("stat:calledwithwrongnumberofarguments"));
returnmake_number(1,result);
}
Thencomestheactualwork.First,thefunctiongetsthearguments.Next,itgetstheinformationforthefile.Ifthecalled
function(lstat()orstat())returnsanerror,thecodesetsERRNOandreturns:
/*fileisfirstarg,arraytoholdresultsissecond*/
if(!get_argument(0,AWK_STRING,&file_param)
||!get_argument(1,AWK_ARRAY,&array_param)){
warning(ext_id,_("stat:badparameters"));
returnmake_number(1,result);
}
if(nargs==3){
statfunc=stat;
}
name=file_param.str_value.str;
array=array_param.array_cookie;
/*alwaysemptyoutthearray*/
clear_array(array);
/*statthefile;iferror,setERRNOandreturn*/
336/479
29/7/2016
TheGNUAwkUsersGuide
ret=statfunc(name,&sbuf);
if(ret<0){
update_ERRNO_int(errno);
}
Thetediousworkisdonebyfill_stat_array(),shownearlier.Whendone,thefunctionreturnstheresultfrom
fill_stat_array():
ret=fill_stat_array(name,array,&sbuf);
}
Finally,itsnecessarytoprovidethegluethatloadsthenewfunction(s)intogawk.
Thefilefuncsextensionalsoprovidesanfts()function,whichweomithere(seeExtensionSampleFileFunctions).
Foritssake,thereisaninitializationfunction:
/*init_filefuncsinitializationroutine*/
staticawk_bool_t
init_filefuncs(void)
{
Wearealmostdone.Weneedanarrayofawk_ext_func_tstructuresforloadingeachfunctionintogawk:
staticawk_ext_func_tfunc_table[]={
{"chdir",do_chdir,1},
{"stat",do_stat,2},
#ifndef__MINGW32__
{"fts",do_fts,3},
#endif
};
Eachextensionmusthavearoutinenameddl_load()toloadeverythingthatneedstobeloaded.Itissimplesttousethe
dl_load_func()macroingawkapi.h:
/*definethedl_load()functionusingtheboilerplatemacro*/
dl_load_func(func_table,filefuncs,"")
Andthatsit!
Previous:InternalFileOps,Up:ExtensionExample[Contents][Index]
16.6.3IntegratingtheExtensions
Nowthatthecodeiswritten,itmustbepossibletoadditatruntimetotherunninggawkinterpreter.First,thecodemust
becompiled.Assumingthatthefunctionsareinafilenamedfilefuncs.c,andidiristhelocationofthegawkapi.h
headerfile,thefollowingsteps106createaGNU/Linuxsharedlibrary:
$gccfPICsharedDHAVE_CONFIG_HcOgIidirfilefuncs.c
$gccofilefuncs.sosharedfilefuncs.o
Oncethelibraryexists,itisloadedbyusingthe@loadkeyword:
#filetestff.awk
@load"filefuncs"
BEGIN{
337/479
29/7/2016
TheGNUAwkUsersGuide
"pwd"|getlinecurdir#savecurrentdirectory
close("pwd")
chdir("/tmp")
system("pwd")#testit
chdir(curdir)#goback
print"Infofortestff.awk"
ret=stat("testff.awk",data)
print"ret=",ret
for(iindata)
printf"data[\"%s\"]=%s\n",i,data[i]
print"testff.awkmodified:",
strftime("%m%d%Y%H:%M:%S",data["mtime"])
print"\nInfoforJUNK"
ret=stat("JUNK",data)
print"ret=",ret
for(iindata)
printf"data[\"%s\"]=%s\n",i,data[i]
print"JUNKmodified:",strftime("%m%d%Y%H:%M:%S",data["mtime"])
}
TheAWKLIBPATHenvironmentvariabletellsgawkwheretofindextensions(seeFindingExtensions).Wesetittothe
currentdirectoryandruntheprogram:
$AWKLIBPATH=$PWDgawkftestff.awk
|/tmp
|Infofortestff.awk
|ret=0
|data["blksize"]=4096
|data["devbsize"]=512
|data["mtime"]=1412004710
|data["mode"]=33204
|data["type"]=file
|data["dev"]=2053
|data["gid"]=1000
|data["ino"]=10358899
|data["ctime"]=1412004710
|data["blocks"]=8
|data["nlink"]=1
|data["name"]=testff.awk
|data["atime"]=1412004716
|data["pmode"]=rwrwr
|data["size"]=666
|data["uid"]=1000
|testff.awkmodified:0929201418:31:50
|
|InfoforJUNK
|ret=1
|JUNKmodified:0101197002:00:00
Next:gawkextlib,Previous:ExtensionExample,Up:DynamicExtensions[Contents][Index]
16.7TheSampleExtensionsinthegawkDistribution
Thissectionprovidesabriefoverviewofthesampleextensionsthatcomeinthegawkdistribution.Someofthemare
intendedforproductionuse(e.g.,thefilefuncs,readdir,andinplaceextensions).Othersmainlyprovideexamplecode
thatshowshowtousetheextensionAPI.
ExtensionSampleFileFunctions:
ExtensionSampleFnmatch:
ExtensionSampleFork:
ExtensionSampleInplace:
Thefilefunctionssample.
Aninterfacetofnmatch().
Aninterfacetofork()andotherprocessfunctions.
Enablinginplacefileediting.
338/479
29/7/2016
TheGNUAwkUsersGuide
ExtensionSampleOrd:
Charactertovaluetocharacterconversions.
ExtensionSampleReaddir:
Aninterfacetoreaddir().
ExtensionSampleRevout:
Reversingoutputsampleoutputwrapper.
ExtensionSampleRev2way:
Reversingdatasampletwowayprocessor.
ExtensionSampleReadwritearray: Serializinganarraytoafile.
ExtensionSampleReadfile:
Readinganentirefileintoastring.
ExtensionSampleTime:
Aninterfacetogettimeofday()andsleep().
ExtensionSampleAPITests:
TestsfortheAPI.
Next:ExtensionSampleFnmatch,Up:ExtensionSamples[Contents][Index]
16.7.1FileRelatedFunctions
Thefilefuncsextensionprovidesthreedifferentfunctions,asfollows.Theusageis:
@load"filefuncs"
Thisishowyouloadtheextension.
result=chdir("/some/directory")
Thechdir()functionisadirecthooktothechdir()systemcalltochangethecurrentdirectory.Itreturnszero
uponsuccessoravaluelessthanzerouponerror.Inthelattercase,itupdatesERRNO.
result=stat("/some/path",statdata[,follow])
Thestat()functionprovidesahookintothestat()systemcall.Itreturnszerouponsuccessoravaluelessthan
zerouponerror.Inthelattercase,itupdatesERRNO.
Bydefault,itusesthelstat()systemcall.However,ifpassedathirdargument,itusesstat()instead.
Inallcases,itclearsthestatdataarray.Whenthecallissuccessful,stat()fillsthestatdataarraywith
informationretrievedfromthefilesystem,asfollows:
Subscript
Fieldinstructstat
"name"
Thefilename
"dev"
st_dev
"ino"
st_ino
"mode"
st_mode
"nlink"
st_nlink
"uid"
st_uid
"gid"
st_gid
"size"
st_size
"atime"
st_atime
"mtime"
st_mtime
"ctime"
st_ctime
"rdev"
st_rdev
"major"
st_major
"minor"
st_minor
"blksize"
st_blksize
"pmode"
"linkval"
Filetype
All
All
All
All
All
All
All
All
All
All
All
Devicefiles
Devicefiles
Devicefiles
All
Ahumanreadableversionofthemodevalue,likethatprintedby
All
ls(forexample,"rwxrxrx")
Thevalueofthesymboliclink
Symboliclinks
339/479
29/7/2016
TheGNUAwkUsersGuide
"type"
Thetypeofthefileasastringoneof"file","blockdev",
"chardev","directory","socket","fifo","symlink","door",or
"unknown"(notallsystemssupportallfiletypes)
All
flags=or(FTS_PHYSICAL,...)
result=fts(pathlist,flags,filedata)
Walkthefiletreesprovidedinpathlistandfillinthefiledataarray,asdescribednext.flagsisthebitwiseOR
ofseveralpredefinedvalues,alsodescribedinamoment.Returnzeroiftherewerenoerrors,otherwisereturn1.
Thefts()functionprovidesahooktotheClibraryfts()routinesfortraversingfilehierarchies.Insteadofreturning
dataaboutonefileatatimeinastream,itfillsinamultidimensionalarraywithdataabouteachfileanddirectory
encounteredintherequestedhierarchies.
Theargumentsareasfollows:
pathlist
Anarrayoffilenames.Theelementvaluesareusedtheindexvaluesareignored.
flags
ThisshouldbethebitwiseORofoneormoreofthefollowingpredefinedconstantflagvalues.Atleastoneof
FTS_LOGICALorFTS_PHYSICALmustbeprovidedotherwisefts()returnsanerrorvalueandsetsERRNO.Theflags
are:
FTS_LOGICAL
Doalogicalfiletraversal,wheretheinformationreturnedforasymboliclinkreferstothelinkedtofile,
andnottothesymboliclinkitself.ThisflagismutuallyexclusivewithFTS_PHYSICAL.
FTS_PHYSICAL
Doaphysicalfiletraversal,wheretheinformationreturnedforasymboliclinkreferstothesymboliclink
itself.ThisflagismutuallyexclusivewithFTS_LOGICAL.
FTS_NOCHDIR
Asaperformanceoptimization,theClibraryfts()routineschangedirectoryastheytraverseafile
hierarchy.Thisflagdisablesthatoptimization.
FTS_COMFOLLOW
Immediatelyfollowasymboliclinknamedinpathlist,whetherornotFTS_LOGICALisset.
FTS_SEEDOT
Bydefault,theClibraryfts()routinesdonotreturnentriesfor.(dot)and..(dotdot).Thisoptioncauses
entriesfordotdottoalsobeincluded.(Theextensionalwaysincludesanentryfordotmoreonthisina
moment.)
FTS_XDEV
Duringatraversal,donotcrossontoadifferentmountedfilesystem.
filedata
Thefiledataarrayholdstheresults.fts()firstclearsit.Thenitcreatesanelementinfiledataforeveryelement
inpathlist.Theindexisthenameofthedirectoryorfilegiveninpathlist.Theelementforthisindexisitselfan
array.Therearetwocases:
Thepathisafile
340/479
29/7/2016
TheGNUAwkUsersGuide
Inthiscase,thearraycontainstwoorthreeelements:
"path"
Thefullpathtothisfile,startingfromtherootthatwasgiveninthepathlistarray.
"stat"
Thiselementisitselfanarray,containingthesameinformationasprovidedbythestat()function
describedearlierforitsstatdataargument.Theelementmaynotbepresentifthestat()systemcall
forthefilefailed.
"error"
Ifsomekindoferrorwasencountered,thearraywillalsocontainanelementnamed"error",which
isastringdescribingtheerror.
Thepathisadirectory
Inthiscase,thearraycontainsoneelementforeachentryinthedirectory.Ifanentryisafile,thatelement
isthesameasforfiles,justdescribed.Iftheentryisadirectory,thatelementis(recursively)anarray
describingthesubdirectory.IfFTS_SEEDOTwasprovidedintheflags,thentherewillalsobeanelement
named"..".Thiselementwillbeanarraycontainingthedataasprovidedbystat().
Inaddition,therewillbeanelementwhoseindexis".".Thiselementisanarraycontainingthesametwo
orthreeelementsasforafile:"path","stat",and"error".
Thefts()functionreturnszeroiftherewerenoerrors.Otherwise,itreturns1.
NOTE:Thefts()extensiondoesnotexactlymimictheinterfaceoftheClibraryfts()routines,choosing
insteadtoprovideaninterfacethatisbasedonassociativearrays,whichismorecomfortabletousefroman
awkprogram.Thisincludesthelackofacomparisonfunction,becausegawkalreadyprovidespowerful
arraysortingfacilities.Althoughanfts_read()likeinterfacecouldhavebeenprovided,thisfeltless
naturalthansimplycreatingamultidimensionalarraytorepresentthefilehierarchyanditsinformation.
Seetest/fts.awkinthegawkdistributionforanexampleuseofthefts()extensionfunction.
Next:ExtensionSampleFork,Previous:ExtensionSampleFileFunctions,Up:ExtensionSamples[Contents][Index]
16.7.2Interfacetofnmatch()
ThisextensionprovidesaninterfacetotheClibraryfnmatch()function.Theusageis:
@load"fnmatch"
result=fnmatch(pattern,string,flags)
Thereturnvalueiszeroonsuccess,FNM_NOMATCHifthestringdidnotmatchthepattern,oradifferentnonzero
valueifanerroroccurred.
Inadditiontothefnmatch()function,thefnmatchextensionaddsoneconstant(FNM_NOMATCH),andanarrayofflagvalues
namedFNM.
Theargumentstofnmatch()are:
pattern
341/479
29/7/2016
TheGNUAwkUsersGuide
Thefilenamewildcardtomatch
string
Thefilenamestring
flag
Eitherzero,orthebitwiseORofoneormoreoftheflagsintheFNMarray
Theflagsareasfollows:
Arrayelement
FNM["CASEFOLD"]
FNM["FILE_NAME"]
FNM["LEADING_DIR"]
FNM["NOESCAPE"]
FNM["PATHNAME"]
FNM["PERIOD"]
Correspondingflagdefinedbyfnmatch()
FNM_CASEFOLD
FNM_FILE_NAME
FNM_LEADING_DIR
FNM_NOESCAPE
FNM_PATHNAME
FNM_PERIOD
Hereisanexample:
@load"fnmatch"
flags=or(FNM["PERIOD"],FNM["NOESCAPE"])
if(fnmatch("*.a","foo.c",flags)==FNM_NOMATCH)
print"nomatch"
Next:ExtensionSampleInplace,Previous:ExtensionSampleFnmatch,Up:ExtensionSamples[Contents][Index]
16.7.3Interfacetofork(),wait(),andwaitpid()
Theforkextensionaddsthreefunctions,asfollows:
@load"fork"
pid=fork()
Thisfunctioncreatesanewprocess.ThereturnvalueiszerointhechildandtheprocessIDnumberofthechild
intheparent,or1uponerror.Inthelattercase,ERRNOindicatestheproblem.Inthechild,PROCINFO["pid"]and
PROCINFO["ppid"]areupdatedtoreflectthecorrectvalues.
ret=waitpid(pid)
Thisfunctiontakesanumericargument,whichistheprocessIDtowaitfor.Thereturnvalueisthatofthe
waitpid()systemcall.
ret=wait()
Thisfunctionwaitsforthefirstchildtodie.Thereturnvalueisthatofthewait()systemcall.
Thereisnocorrespondingexec()function.
Hereisanexample:
@load"fork"
if((pid=fork())==0)
342/479
29/7/2016
TheGNUAwkUsersGuide
print"hellofromthechild"
else
print"hellofromtheparent"
Next:ExtensionSampleOrd,Previous:ExtensionSampleFork,Up:ExtensionSamples[Contents][Index]
16.7.4EnablingInPlaceFileEditing
TheinplaceextensionemulatesGNUsedsioption,whichperformsinplaceeditingofeachinputfile.Itusesthe
bundledinplace.awkincludefiletoinvoketheextensionproperly:
#inplaceloadandinvoketheinplaceextension.
@load"inplace"
#PleasesetINPLACE_SUFFIXtomakeabackupcopy.Forexample,youmay
#wanttosetINPLACE_SUFFIXto.bakonthecommandlineorinaBEGINrule.
#N.B.Wecallinplace_end()intheBEGINFILEandENDrulessothatany
#actionsinanENDFILErulewillberedirectedasexpected.
BEGINFILE{
if(_inplace_filename!="")
inplace_end(_inplace_filename,INPLACE_SUFFIX)
inplace_begin(_inplace_filename=FILENAME,INPLACE_SUFFIX)
}
END{
inplace_end(FILENAME,INPLACE_SUFFIX)
}
Foreachregularfilethatisprocessed,theextensionredirectsstandardoutputtoatemporaryfileconfiguredtohavethe
sameownerandpermissionsastheoriginal.Afterthefilehasbeenprocessed,theextensionrestoresstandardoutputto
itsoriginaldestination.IfINPLACE_SUFFIXisnotanemptystring,theoriginalfileislinkedtoabackupfilenamecreated
byappendingthatsuffix.Finally,thetemporaryfileisrenamedtotheoriginalfilename.
The_inplace_filenamevariableservestokeeptrackofthecurrentfilenamesoastonotinvokeinplace_end()before
processingthefirstfile.
Ifanyerroroccurs,theextensionissuesafatalerrortoterminateprocessingimmediatelywithoutdamagingtheoriginal
file.
Herearesomesimpleexamples:
$gawkiinplace'{gsub(/foo/,"bar")};{print}'file1file2file3
Tokeepabackupcopyoftheoriginalfiles,trythis:
$gawkiinplacevINPLACE_SUFFIX=.bak'{gsub(/foo/,"bar")}
>{print}'file1file2file3
Next:ExtensionSampleReaddir,Previous:ExtensionSampleInplace,Up:ExtensionSamples[Contents][Index]
16.7.5CharacterandNumericvalues:ord()andchr()
Theordchrextensionaddstwofunctions,namedord()andchr(),asfollows:
@load"ordchr"
343/479
29/7/2016
TheGNUAwkUsersGuide
number=ord(string)
Returnthenumericvalueofthefirstcharacterinstring.
char=chr(number)
Returnastringwhosefirstcharacteristhatrepresentedbynumber.
ThesefunctionsareinspiredbythePascallanguagefunctionsofthesamename.Hereisanexample:
@load"ordchr"
printf("Thenumericvalueof'A'is%d\n",ord("A"))
printf("Thestringvalueof65is%s\n",chr(65))
Next:ExtensionSampleRevout,Previous:ExtensionSampleOrd,Up:ExtensionSamples[Contents][Index]
16.7.6ReadingDirectories
Thereaddirextensionaddsaninputparserfordirectories.Theusageisasfollows:
@load"readdir"
Whenthisextensionisinuse,insteadofskippingdirectoriesnamedonthecommandline(orwithgetline),theyare
read,witheachentryreturnedasarecord.
Therecordconsistsofthreefields.Thefirsttwoaretheinodenumberandthefilename,separatedbyaforwardslash
character.Onsystemswherethedirectoryentrycontainsthefiletype,therecordhasathirdfield(alsoseparatedbya
slash),whichisasingleletterindicatingthetypeofthefile.Thelettersandtheircorrespondingfiletypesareshownin
Table16.2.
Letter
b
c
d
f
l
p
s
u
Filetype
Blockdevice
Characterdevice
Directory
Regularfile
Symboliclink
Namedpipe(FIFO)
Socket
Anythingelse(unknown)
Table16.2:Filetypesreturnedbythereaddirextension
Onsystemswithoutthefiletypeinformation,thethirdfieldisalwaysu.
NOTE:OnGNU/Linuxsystems,therearefilesystemsthatdontsupportthed_typeentry(seethe
readdir(3)manualpage),andsothefiletypeisalwaysu.Youcanusethefilefuncsextensiontocall
stat()inordertogetcorrecttypeinformation.
Hereisanexample:
@load"readdir"
BEGIN{FS="/"}
{print"filenameis",$2}
Next:ExtensionSampleRev2way,Previous:ExtensionSampleReaddir,Up:ExtensionSamples[Contents][Index]
344/479
29/7/2016
TheGNUAwkUsersGuide
16.7.7ReversingOutput
Therevoutputextensionaddsasimpleoutputwrapperthatreversesthecharactersineachoutputline.Itsmainpurpose
istoshowhowtowriteanoutputwrapper,althoughitmaybemildlyamusingfortheunwary.Hereisanexample:
@load"revoutput"
BEGIN{
REVOUT=1
print"don'tpanic">"/dev/stdout"
}
Theoutputfromthisprogramiscinapt'nod.
Next:ExtensionSampleReadwritearray,Previous:ExtensionSampleRevout,Up:ExtensionSamples[Contents]
[Index]
16.7.8TwoWayI/OExample
Therevtwowayextensionaddsasimpletwowayprocessorthatreversesthecharactersineachlinesenttoitforreading
backbytheawkprogram.Itsmainpurposeistoshowhowtowriteatwowayprocessor,althoughitmayalsobemildly
amusing.Thefollowingexampleshowshowtouseit:
@load"revtwoway"
BEGIN{
cmd="/magic/mirror"
print"don'tpanic"|&cmd
cmd|&getlineresult
printresult
close(cmd)
}
Theoutputfromthisprogramalsois:cinapt'nod.
Next:ExtensionSampleReadfile,Previous:ExtensionSampleRev2way,Up:ExtensionSamples[Contents][Index]
16.7.9DumpingandRestoringanArray
Therwarrayextensionaddstwofunctions,namedwritea()andreada(),asfollows:
@load"rwarray"
ret=writea(file,array)
Thisfunctiontakesastringargument,whichisthenameofthefiletowhichtodumpthearray,andthearrayitself
asthesecondargument.writea()understandsarraysofarrays.Itreturnsoneonsuccess,orzerouponfailure.
ret=reada(file,array)
reada()istheinverseofwritea()itreadsthefilenamedasitsfirstargument,fillinginthearraynamedasthe
secondargument.Itclearsthearrayfirst.Heretoo,thereturnvalueisoneonsuccess,orzerouponfailure.
Thearraycreatedbyreada()isidenticaltothatwrittenbywritea()inthesensethatthecontentsarethesame.
However,duetoimplementationissues,thearraytraversalorderoftherecreatedarrayislikelytobedifferentfromthat
oftheoriginalarray.Asarraytraversalorderinawkisbydefaultundefined,thisis(technically)notaproblem.Ifyou
needtoguaranteeaparticulartraversalorder,usethearraysortingfeaturesingawktodoso(seeArraySorting).
345/479
29/7/2016
TheGNUAwkUsersGuide
Thefilecontainsbinarydata.Allintegralvaluesarewritteninnetworkbyteorder.However,doubleprecisionfloating
pointvaluesarewrittenasnativebinarydata.Thus,arrayscontainingonlystringdatacantheoreticallybedumpedon
systemswithonebyteorderandrestoredonsystemswithadifferentone,butthishasnotbeentried.
Hereisanexample:
@load"rwarray"
ret=writea("arraydump.bin",array)
ret=reada("arraydump.bin",array)
Next:ExtensionSampleTime,Previous:ExtensionSampleReadwritearray,Up:ExtensionSamples[Contents]
[Index]
16.7.10ReadinganEntireFile
Thereadfileextensionaddsasinglefunctionnamedreadfile(),andaninputparser:
@load"readfile"
result=readfile("/some/path")
Theargumentisthenameofthefiletoread.Thereturnvalueisastringcontainingtheentirecontentsofthe
requestedfile.Uponerror,thefunctionreturnstheemptystringandsetsERRNO.
BEGIN{PROCINFO["readfile"]=1}
Inaddition,theextensionaddsaninputparserthatisactivatedifPROCINFO["readfile"]exists.Whenactivated,
eachinputfileisreturnedinitsentiretyas$0.RTissettothenullstring.
Hereisanexample:
@load"readfile"
contents=readfile("/path/to/file");
if(contents==""&&ERRNO!=""){
print("problemreadingfile",ERRNO)>"/dev/stderr"
...
}
Next:ExtensionSampleAPITests,Previous:ExtensionSampleReadfile,Up:ExtensionSamples[Contents][Index]
16.7.11ExtensionTimeFunctions
Thetimeextensionaddstwofunctions,namedgettimeofday()andsleep(),asfollows:
@load"time"
the_time=gettimeofday()
Returnthetimeinsecondsthathaselapsedsince19700101UTCasafloatingpointvalue.Ifthetimeis
unavailableonthisplatform,return1andsetERRNO.Thereturnedtimeshouldhavesubsecondprecision,butthe
actualprecisionmayvarybasedontheplatform.IfthestandardCgettimeofday()systemcallisavailableonthis
platform,thenitsimplyreturnsthevalue.Otherwise,ifonMSWindows,ittriestouse
GetSystemTimeAsFileTime().
346/479
29/7/2016
TheGNUAwkUsersGuide
result=sleep(seconds)
Attempttosleepforsecondsseconds.Ifsecondsisnegative,ortheattempttosleepfails,return1andsetERRNO.
Otherwise,returnzeroaftersleepingfortheindicatedamountoftime.Notethatsecondsmaybeafloatingpoint
(nonintegral)value.Implementationdetails:dependingonplatformavailability,thisfunctiontriestouse
nanosleep()orselect()toimplementthedelay.
Previous:ExtensionSampleTime,Up:ExtensionSamples[Contents][Index]
16.7.12APITests
ThetestextextensionexercisespartsoftheextensionAPIthatarenottestedbytheothersamples.The
extension/testext.cfilecontainsboththeCcodefortheextensionandawktestcodeinsideCcommentsthatrunthe
tests.Thetestingframeworkextractstheawkcodeandrunsthetests.Seethesourcefileformoreinformation.
Next:Extensionsummary,Previous:ExtensionSamples,Up:DynamicExtensions[Contents][Index]
16.8ThegawkextlibProject
Thegawkextlibprojectprovidesanumberofgawkextensions,includingoneforprocessingXMLfiles.Thisisthe
evolutionoftheoriginalxgawk(XMLgawk)project.
Asofthiswriting,therearesevenextensions:
errnoextension
GDgraphicslibraryextension
PDFextension
PostgreSQLextension
MPFRlibraryextension(thisprovidesaccesstoanumberofMPFRfunctionsthatgawksnativeMPFRsupport
doesnot)
Redisextension
XMLparserextension,usingtheExpatXMLparsinglibrary
YoucancheckoutthecodeforthegawkextlibprojectusingtheGitdistributedsourcecodecontrolsystem.The
commandisasfollows:
gitclonegit://git.code.sf.net/p/gawkextlib/codegawkextlibcode
YouwillneedtohavetheExpatXMLparserlibraryinstalledinordertobuildandusetheXMLextension.
Inaddition,youmusthavetheGNUAutotoolsinstalled(Autoconf,Automake,Libtool,andGNUgettext).
Thesimplerecipeforbuildingandtestinggawkextlibisasfollows.First,buildandinstallgawk:
cd.../path/to/gawk/code
./configureprefix=/tmp/newgawkInstallin/tmp/newgawkfornow
make&&makecheckBuildandcheckthatallisOK
makeinstallInstallgawk
Next,gotohttp://sourceforge.net/projects/gawkextlib/filestodownloadgawkextlibandanyextensionsthatyouwould
liketobuild.TheREADMEfileatthatsiteexplainshowtobuildthecode.Ifyouinstalledgawkinanonstandardlocation,
youwillneedtospecify./configurewithgawk=/path/to/gawktofindit.Youmayneedtousethesudoutilityto
installbothgawkandgawkextlib,dependinguponhowyoursystemworks.
Ifyouwriteanextensionthatyouwishtosharewithothergawkusers,considerdoingsothroughthegawkextlibproject.
Seetheprojectswebsiteformoreinformation.
347/479
29/7/2016
TheGNUAwkUsersGuide
Next:ExtensionExercises,Previous:gawkextlib,Up:DynamicExtensions[Contents][Index]
16.9Summary
Youcanwriteextensions(sometimescalledplugins)forgawkinCorC++usingtheapplicationprogramming
interface(API)definedbythegawkdevelopers.
ExtensionsmusthavealicensecompatiblewiththeGNUGeneralPublicLicense(GPL),andtheymustassert
thatfactbydeclaringavariablenamedplugin_is_GPL_compatible.
Communicationbetweengawkandanextensionistwoway.gawkpassesastructtotheextensionthatcontains
variousdatafieldsandfunctionpointers.Theextensioncanthencallintogawkviathesuppliedfunctionpointers
toaccomplishcertaintasks.
Oneofthesetasksistoregisterthenameandimplementationofnewawklevelfunctionswithgawk.The
implementationtakestheformofaCfunctionpointerwithadefinedsignature.Byconvention,implementation
functionsarenameddo_XXXX()forsomeawklevelfunctionXXXX().
TheAPIisdefinedinaheaderfilenamedgawkapi.h.Youmustincludeanumberofstandardheaderfilesbefore
includingitinyoursourcefile.
APIfunctionpointersareprovidedforthefollowingkindsofoperations:
Allocating,reallocating,andreleasingmemory
Registrationfunctions(youmayregisterextensionfunctions,exitcallbacks,aversionstring,inputparsers,
outputwrappers,andtwowayprocessors)
Printingfatal,warning,andlintwarningmessages
UpdatingERRNO,orunsettingit
Accessingparameters,includingconvertinganundefinedparameterintoanarray
Symboltableaccess(retrievingaglobalvariable,creatingone,orchangingone)
Creatingandreleasingcachedvaluesthisprovidesanefficientwaytousevaluesformultiplevariablesand
canbeabigperformancewin
Manipulatingarrays(retrieving,adding,deleting,andmodifyingelementsgettingthecountofelementsin
anarraycreatinganewarrayclearinganarrayandflatteninganarrayforeasyCstyleloopingoverallits
indicesandelements)
TheAPIdefinesanumberofstandarddatatypesforrepresentingawkvalues,arrayelements,andarrays.
TheAPIprovidesconveniencefunctionsforconstructingvalues.Italsoprovidesmemorymanagementfunctions
toensurecompatibilitybetweenmemoryallocatedbygawkandmemoryallocatedbyanextension.
Allmemorypassedfromgawktoanextensionmustbetreatedasreadonlybytheextension.
AllmemorypassedfromanextensiontogawkmustcomefromtheAPIsmemoryallocationfunctions.gawktakes
responsibilityforthememoryandreleasesitwhenappropriate.
TheAPIprovidesinformationabouttherunningversionofgawksothatanextensioncanmakesureitis
compatiblewiththegawkthatloadedit.
Itiseasiesttostartanewextensionbycopyingtheboilerplatecodedescribedinthischapter.Macrosinthe
gawkapi.hheaderfilemakethiseasiertodo.
Thegawkdistributionincludesanumberofsmallbutusefulsampleextensions.Thegawkextlibprojectincludes
severalmore(larger)extensions.Ifyouwishtowriteanextensionandcontributeittothecommunityofgawk
users,thegawkextlibprojectistheplacetodoso.
Previous:Extensionsummary,Up:DynamicExtensions[Contents][Index]
16.10Exercises
1.Addfunctionstoimplementsystemcallssuchaschown(),chmod(),andumask()tothefileoperationsextension
presentedinInternalFileOps.
2.Writeaninputparserthatprintsapromptiftheinputisafromaterminaldevice.Youcanusetheisatty()
functiontotelliftheinputfileisaterminal.(Hint:thisfunctionisusuallyexpensivetocalltrytocallitjust
once.)Thecontentofthepromptshouldcomefromavariablesettablebyawklevelcode.Youcanwritethe
prompttostanarderror.However,forbestresults,openanewfiledescriptor(orfilepointer)on/dev/ttyand
printthepromptthere,incasestandarderrorhasbeenredirected.
Whyisstandarderrorabetterchoicethanstandardoutputforwritingtheprompt?Whichreadingmechanism
348/479
29/7/2016
TheGNUAwkUsersGuide
shouldyoureplace,theonetogetarecord,ortheonetoreadrawbytes?
3.(Hard.)Howwouldyouprovidenamespacesingawk,sothatthenamesoffunctionsindifferentextensionsdont
conflictwitheachother?Ifyoucomeupwithareallygoodscheme,contactthegawkmaintainertotellhimabout
it.
4.Writeawrapperscriptthatprovidesaninterfacesimilartosedifortheinplaceextensionpresentedin
ExtensionSampleInplace.
Next:Installation,Previous:DynamicExtensions,Up:Top[Contents][Index]
PartIV:
Appendices
ThisWebpagedescribestheGNUimplementationofawk,whichfollowsthePOSIXspecification.Manylongtimeawk
userslearnedawkprogrammingwiththeoriginalawkimplementationinVersion7Unix.(Thisimplementationwasthe
basisforawkinBerkeleyUnix,through4.3Reno.SubsequentversionsofBerkeleyUnix,and,forawhile,some
systemsderivedfrom4.4BSDLite,usedvariousversionsofgawkfortheirawk.)Thischapterbrieflydescribesthe
evolutionoftheawklanguage,withcrossreferencestootherpartsoftheWebpagewhereyoucanfindmore
information.
V7/SVR3.1:
ThemajorchangesbetweenV7andSystemVRelease3.1.
SVR4:
MinorchangesbetweenSystemVReleases3.1and4.
POSIX:
NewfeaturesfromthePOSIXstandard.
BTL:
NewfeaturesfromBrianKernighansversionofawk.
POSIX/GNU:
TheextensionsingawknotinPOSIXawk.
FeatureHistory:
Thehistoryofthefeaturesingawk.
CommonExtensions: CommonExtensionsSummary.
RangesandLocales: Howlocalesusedtoaffectregexpranges.
Contributors:
Themajorcontributorstogawk.
Historysummary:
Historysummary.
Next:SVR4,Up:LanguageHistory[Contents][Index]
A.1MajorChangesBetweenV7andSVR3.1
TheawklanguageevolvedconsiderablybetweenthereleaseofVersion7Unix(1978)andthenewversionthatwasfirst
madegenerallyavailableinSystemVRelease3.1(1987).Thissectionsummarizesthechanges,withcrossreferences
tofurtherdetails:
Therequirementfor;toseparaterulesonaline(seeStatements/Lines)
Userdefinedfunctionsandthereturnstatement(seeUserdefined)
Thedeletestatement(seeDelete).
Thedowhilestatement(seeDoStatement)
Thebuiltinfunctionsatan2(),cos(),sin(),rand(),andsrand()(seeNumericFunctions)
Thebuiltinfunctionsgsub(),sub(),andmatch()(seeStringFunctions)
Thebuiltinfunctionsclose()andsystem()(seeI/OFunctions)
TheARGC,ARGV,FNR,RLENGTH,RSTART,andSUBSEPpredefinedvariables(seeBuiltinVariables)
Assignable$0(seeChangingFields)
Theconditionalexpressionusingtheternaryoperator?:(seeConditionalExp)
349/479
29/7/2016
TheGNUAwkUsersGuide
Theexpressionindxinarrayoutsideofforstatements(seeReferencetoElements)
Theexponentiationoperator^(seeArithmeticOps)anditsassignmentoperatorform^=(seeAssignmentOps)
Ccompatibleoperatorprecedence,whichbreakssomeoldawkprograms(seePrecedence)
RegexpsasthevalueofFS(seeFieldSeparators)andasthethirdargumenttothesplit()function(seeString
Functions),ratherthanusingonlythefirstcharacterofFS
Dynamicregexpsasoperandsofthe~and!~operators(seeComputedRegexps)
Theescapesequences\b,\f,and\r(seeEscapeSequences)
Redirectionofinputforthegetlinefunction(seeGetline)
MultipleBEGINandENDrules(seeBEGIN/END)
Multidimensionalarrays(seeMultidimensional)
Next:POSIX,Previous:V7/SVR3.1,Up:LanguageHistory[Contents][Index]
A.2ChangesBetweenSVR3.1andSVR4
TheSystemVRelease4(1989)versionofUnixawkaddedthesefeatures(someofwhichoriginatedingawk):
TheENVIRONarray(seeBuiltinVariables)
Multiplefoptionsonthecommandline(seeOptions)
Thevoptionforassigningvariablesbeforeprogramexecutionbegins(seeOptions)
Thesignalforterminatingcommandlineoptions
The\a,\v,and\xescapesequences(seeEscapeSequences)
Adefinedreturnvalueforthesrand()builtinfunction(seeNumericFunctions)
Thetoupper()andtolower()builtinstringfunctionsforcasetranslation(seeStringFunctions)
Acleanerspecificationforthe%cformatcontrolletterintheprintffunction(seeControlLetters)
Theabilitytodynamicallypassthefieldwidthandprecision("%*.*d")intheargumentlistofprintfand
sprintf()(seeControlLetters)
Theuseofregexpconstants,suchas/foo/,asexpressions,wheretheyareequivalenttousingthematching
operator,asin$0~/foo/(seeUsingConstantRegexps)
Processingofescapesequencesinsidecommandlinevariableassignments(seeAssignmentOptions)
Next:BTL,Previous:SVR4,Up:LanguageHistory[Contents][Index]
A.3ChangesBetweenSVR4andPOSIXawk
ThePOSIXCommandLanguageandUtilitiesstandardforawk(1992)introducedthefollowingchangesintothe
language:
TheuseofWforimplementationspecificoptions(seeOptions)
TheuseofCONVFMTforcontrollingtheconversionofnumberstostrings(seeConversion)
Theconceptofanumericstringandtightercomparisonrulestogowithit(seeTypingandComparison)
Theuseofpredefinedvariablesasfunctionparameternamesisforbidden(seeDefinitionSyntax)
Morecompletedocumentationofmanyofthepreviouslyundocumentedfeaturesofthelanguage
In2012,anumberofextensionsthathadbeencommonlyavailableformanyyearswerefinallyaddedtoPOSIX.They
are:
Thefflush()builtinfunctionforflushingbufferedoutput(seeI/OFunctions)
Thenextfilestatement(seeNextfileStatement)
Theabilitytodeleteallofanarrayatoncewithdeletearray(seeDelete)
SeeCommonExtensions,foralistofcommonextensionsnotpermittedbythePOSIXstandard.
The2008POSIXstandardcanbefoundonlineathttp://www.opengroup.org/onlinepubs/9699919799/.
350/479
29/7/2016
TheGNUAwkUsersGuide
Next:POSIX/GNU,Previous:POSIX,Up:LanguageHistory[Contents][Index]
A.4ExtensionsinBrianKernighansawk
BrianKernighanhasmadehisversionavailableviahishomepage(seeOtherVersions).
Thissectiondescribescommonextensionsthatoriginallyappearedinhisversionofawk:
The**and**=operators(seeArithmeticOpsandAssignmentOps)
Theuseoffuncasanabbreviationforfunction(seeDefinitionSyntax)
Thefflush()builtinfunctionforflushingbufferedoutput(seeI/OFunctions)
SeeCommonExtensions,forafulllistoftheextensionsavailableinhisawk.
Next:FeatureHistory,Previous:BTL,Up:LanguageHistory[Contents][Index]
A.5ExtensionsingawkNotinPOSIXawk
TheGNUimplementation,gawk,addsalargenumberoffeatures.Theycanallbedisabledwitheitherthetraditional
orposixoptions(seeOptions).
Anumberoffeatureshavecomeandgoneovertheyears.ThissectionsummarizestheadditionalfeaturesoverPOSIX
awkthatareinthecurrentversionofgawk.
Additionalpredefinedvariables:
TheARGIND,BINMODE,ERRNO,FIELDWIDTHS,FPAT,IGNORECASE,LINT,PROCINFO,RT,andTEXTDOMAINvariables
(seeBuiltinVariables)
SpecialfilesinI/Oredirections:
The/dev/stdin,/dev/stdout,/dev/stderr,and/dev/fd/Nspecialfilenames(seeSpecialFiles)
The/inet,/inet4,and/inet6specialfilesforTCP/IPnetworkingusing|&tospecifywhichversionof
theIPprotocoltouse(seeTCP/IPNetworking)
Changesand/oradditionstothelanguage:
The\xescapesequence(seeEscapeSequences)
FullsupportforbothPOSIXandGNUregexps(seeRegexp)
TheabilityforFSandforthethirdargumenttosplit()tobenullstrings(seeSingleCharacterFields)
TheabilityforRStobearegexp(seeRecords)
Theabilitytouseoctalandhexadecimalconstantsinawkprogramsourcecode(seeNondecimalnumbers)
The|&operatorfortwowayI/Otoacoprocess(seeTwowayI/O)
Indirectfunctioncalls(seeIndirectCalls)
Directoriesonthecommandlineproduceawarningandareskipped(seeCommandlinedirectories)
Newkeywords:
TheBEGINFILEandENDFILEspecialpatterns(seeBEGINFILE/ENDFILE)
Theswitchstatement(seeSwitchStatement)
Changestostandardawkfunctions:
Theoptionalsecondargumenttoclose()thatallowsclosingoneendofatwowaypipetoacoprocess
(seeTwowayI/O)
POSIXcomplianceforgsub()andsub()withposix
Thelength()functionacceptsanarrayargumentandreturnsthenumberofelementsinthearray(see
StringFunctions)
Theoptionalthirdargumenttothematch()functionforcapturingtextmatchingsubexpressionswithina
regexp(seeStringFunctions)
Positionalspecifiersinprintfformatsformakingtranslationseasier(seePrintfOrdering)
Thesplit()functionsadditionaloptionalfourthargument,whichisanarraytoholdthetextofthefield
separators(seeStringFunctions)
Additionalfunctionsonlyingawk:
Thegensub(),patsplit(),andstrtonum()functionsformorepowerfultextmanipulation(seeString
Functions)
351/479
29/7/2016
TheGNUAwkUsersGuide
Theasort()andasorti()functionsforsortingarrays(seeArraySorting)
Themktime(),systime(),andstrftime()functionsforworkingwithtimestamps(seeTimeFunctions)
Theand(),compl(),lshift(),or(),rshift(),andxor()functionsforbitmanipulation(seeBitwise
Functions)
Theisarray()functiontocheckifavariableisanarrayornot(seeTypeFunctions)
Thebindtextdomain(),dcgettext(),anddcngettext()functionsforinternationalization(seeProgrammer
i18n)
Changesand/oradditionsinthecommandlineoptions:
TheAWKPATHenvironmentvariableforspecifyingapathsearchforthefcommandlineoption(see
Options)
TheAWKLIBPATHenvironmentvariableforspecifyingapathsearchforthelcommandlineoption(see
Options)
Theb,c,C,d,D,e,E,g,h,i,l,L,M,n,N,o,O,p,P,r,S,t,andVshortoptions.
Also,theabilitytouseGNUstylelongnamedoptionsthatstartwith,andtheassign,bignum,
charactersasbytes,copyright,debug,dumpvariables,exec,fieldseparator,file,gen
pot,help,include,lint,lintold,load,nondecimaldata,optimize,posix,pretty
print,profile,reinterval,sandbox,source,traditional,uselcnumeric,andversion
longoptions(seeOptions).
Supportforthefollowingobsoletesystemswasremovedfromthecodeandthedocumentationforgawkversion
4.0:
Amiga
Atari
BeOS
Cray
MIPSRiscOS
MSDOSwiththeMicrosoftCompiler
MSWindowswiththeMicrosoftCompiler
NeXT
SunOS3.x,Sun386(RoadRunner)
Tandem(nonPOSIX)
PrestandardVAXCcompilerforVAX/VMS
GCCforVAXandAlphahasnotbeentestedforawhile.
Supportforthefollowingobsoletesystemwasremovedfromthecodeforgawkversion4.1:
Ultrix
Next:CommonExtensions,Previous:POSIX/GNU,Up:LanguageHistory[Contents][Index]
A.6HistoryofgawkFeatures
ThissectiondescribesthefeaturesingawkoverandabovethoseinPOSIXawk,intheordertheywereaddedtogawk.
Version2.10ofgawkintroducedthefollowingfeatures:
TheAWKPATHenvironmentvariableforspecifyingapathsearchforthefcommandlineoption(seeOptions).
TheIGNORECASEvariableanditseffects(seeCasesensitivity).
The/dev/stdin,/dev/stdout,/dev/stderrand/dev/fd/Nspecialfilenames(seeSpecialFiles).
TheFIELDWIDTHSvariableanditseffects(seeConstantSize).
Thesystime()andstrftime()builtinfunctionsforobtainingandprintingtimestamps(seeTimeFunctions).
Additionalcommandlineoptions(seeOptions):
TheWlintoptiontoprovideerrorandportabilitycheckingforboththesourcecodeandatruntime.
TheWcompatoptiontoturnofftheGNUextensions.
TheWposixoptionforfullPOSIXcompliance.
Version2.14ofgawkintroducedthefollowingfeature:
352/479
29/7/2016
TheGNUAwkUsersGuide
Thenextfilestatementforskippingtothenextdatafile(seeNextfileStatement).
Newvariables(seeBuiltinVariables):
ARGIND,whichtracksthemovementofFILENAMEthroughARGV.
ERRNO,whichcontainsthesystemerrormessagewhengetlinereturns1orclose()fails.
The/dev/pid,/dev/ppid,/dev/pgrpid,and/dev/userspecialfilenames.Thesehavesincebeenremoved.
Theabilitytodeleteallofanarrayatoncewithdeletearray(seeDelete).
Commandlineoptionchanges(seeOptions):
TheabilitytouseGNUstylelongnamedoptionsthatstartwith.
Thesourceoptionformixingcommandlineandlibraryfilesourcecode.
Neworchangedvariables:
IGNORECASEchanged,nowapplyingtostringcomparisonaswellasregexpoperations(seeCase
sensitivity).
RT,whichcontainstheinputtextthatmatchedRS(seeRecords).
FullsupportforbothPOSIXandGNUregexps(seeRegexp).
Thegensub()functionformorepowerfultextmanipulation(seeStringFunctions).
Thestrftime()functionacquiredadefaulttimeformat,allowingittobecalledwithnoarguments(seeTime
Functions).
TheabilityforFSandforthethirdargumenttosplit()tobenullstrings(seeSingleCharacterFields).
TheabilityforRStobearegexp(seeRecords).
Thenextfilestatementbecamenextfile(seeNextfileStatement).
Thefflush()functionfromBWKawk(thenatBellLaboratoriesseeI/OFunctions).
Newcommandlineoptions:
ThelintoldoptiontowarnaboutconstructsthatarenotavailableintheoriginalVersion7Unix
versionofawk(seeV7/SVR3.1).
ThemoptionfromBWKawk.(BrianwasstillatBellLaboratoriesatthetime.)Thiswaslaterremoved
frombothhisawkandfromgawk.
Thereintervaloptiontoprovideintervalexpressionsinregexps(seeRegexpOperators).
Thetraditionaloptionwasaddedasabetternameforcompat(seeOptions).
TheuseofGNUAutoconftocontroltheconfigurationprocess(seeQuickInstallation).
Amigasupport.Thishassincebeenremoved.
Newvariables(seeBuiltinVariables):
BINMODE,fornonPOSIXsystems,whichallowsbinaryI/Oforinputand/oroutputfiles(seePCUsing).
LINT,whichdynamicallycontrolslintwarnings.
PROCINFO,anarrayforprovidingprocessrelatedinformation.
TEXTDOMAIN,forsettinganapplicationsinternationalizationtextdomain(seeInternationalization).
Theabilitytouseoctalandhexadecimalconstantsinawkprogramsourcecode(seeNondecimalnumbers).
The|&operatorfortwowayI/Otoacoprocess(seeTwowayI/O).
The/inetspecialfilesforTCP/IPnetworkingusing|&(seeTCP/IPNetworking).
Theoptionalsecondargumenttoclose()thatallowsclosingoneendofatwowaypipetoacoprocess(seeTwo
wayI/O).
Theoptionalthirdargumenttothematch()functionforcapturingtextmatchingsubexpressionswithinaregexp
(seeStringFunctions).
Positionalspecifiersinprintfformatsformakingtranslationseasier(seePrintfOrdering).
Anumberofnewbuiltinfunctions:
Theasort()andasorti()functionsforsortingarrays(seeArraySorting).
Thebindtextdomain(),dcgettext()anddcngettext()functionsforinternationalization(seeProgrammer
i18n).
Theextension()functionandtheabilitytoaddnewbuiltinfunctionsdynamically(seeDynamic
Extensions).
353/479
29/7/2016
TheGNUAwkUsersGuide
Themktime()functionforcreatingtimestamps(seeTimeFunctions).
Theand(),or(),xor(),compl(),lshift(),rshift(),andstrtonum()functions(seeBitwiseFunctions).
Thesupportfornextfileastwowordswasremovedcompletely(seeNextfileStatement).
Additionalcommandlineoptions(seeOptions):
Thedumpvariablesoptiontoprintalistofallglobalvariables.
Theexecoption,foruseinCGIscripts.
Thegenpocommandlineoptionandtheuseofaleadingunderscoretomarkstringsthatshouldbe
translated(seeStringExtraction).
Thenondecimaldataoptiontoallownondecimalinputdata(seeNondecimalData).
Theprofileoptionandpgawk,theprofilingversionofgawk,forproducingexecutionprofilesofawk
programs(seeProfiling).
Theuselcnumericoptiontoforcegawktousethelocalesdecimalpointforparsinginputdata(see
Conversion).
TheuseofGNUAutomaketohelpinstandardizingtheconfigurationprocess(seeQuickInstallation).
TheuseofGNUgettextforgawksownmessageoutput(seeGawkI18N).
BeOSsupport.Thiswaslaterremoved.
Tandemsupport.Thiswaslaterremoved.
TheAtariportbecameofficiallyunsupportedandwaslaterremovedentirely.
ThesourcecodechangedtouseISOCstandardstylefunctiondefinitions.
POSIXcomplianceforsub()andgsub()(seeGoryDetails).
Thelength()functionwasextendedtoacceptanarrayargumentandreturnthenumberofelementsinthearray
(seeStringFunctions).
Thestrftime()functionacquiredathirdargumenttoenableprintingtimesasUTC(seeTimeFunctions).
Variableadditions:
FPAT,whichallowsyoutospecifyaregexpthatmatchesthefields,insteadofmatchingthefieldseparator
(seeSplittingByContent).
IfPROCINFO["sorted_in"]exists,for(iggyinfoo)loopssorttheindicesbeforeloopingoverthem.The
valueofthiselementprovidescontroloverhowtheindicesaresortedbeforethelooptraversalstarts(see
ControllingScanning).
PROCINFO["strftime"],whichholdsthedefaultformatforstrftime()(seeTimeFunctions).
Thespecialfiles/dev/pid,/dev/ppid,/dev/pgrpidand/dev/userwereremoved.
SupportforIPv6wasaddedviathe/inet6specialfile./inet4forcesIPv4and/inetchoosesthesystemdefault,
whichisprobablyIPv4(seeTCP/IPNetworking).
Theuseof\sand\Sescapesequencesinregularexpressions(seeGNURegexpOperators).
Intervalexpressionsbecamepartofdefaultregularexpressions(seeRegexpOperators).
POSIXcharacterclassesworkevenwithtraditional(seeRegexpOperators).
breakandcontinuebecameinvalidoutsidealoop,evenwithtraditional(seeBreakStatement,andalsosee
ContinueStatement).
fflush(),nextfile,anddeletearrayareallowedif posixortraditional,sincetheyareallnowpartof
POSIX.
Anoptionalthirdargumenttoasort()andasorti(),specifyinghowtosort(seeStringFunctions).
Thebehavioroffflush()changedtomatchBWKawkandforPOSIXnowbothfflush()andfflush("")
flushallopenoutputredirections(seeI/OFunctions).
Theisarray()functionwhichdistinguishesifanitemisanarrayornot,tomakeitpossibletotraversearraysof
arrays(seeTypeFunctions).
Thepatsplit()functionwhichgivesthesamecapabilityasFPAT,forsplitting(seeStringFunctions).
Anoptionalfourthargumenttothesplit()function,whichisanarraytoholdthevaluesoftheseparators(see
StringFunctions).
Arraysofarrays(seeArraysofArrays).
TheBEGINFILEandENDFILEspecialpatterns(seeBEGINFILE/ENDFILE).
Indirectfunctioncalls(seeIndirectCalls).
switch/caseareenabledbydefault(seeSwitchStatement).
Thebandcharactersasbytesoptionswhichpreventgawkfromtreatinginputasamultibytestring.
Theredundantcompat,copyleft,andusagelongoptionswereremoved.
354/479
29/7/2016
TheGNUAwkUsersGuide
Thegenpooptionwasfinallyrenamedtothecorrectgenpot.
Thesandboxoptionwhichdisablescertainfeatures.
Alllongoptionsacquiredcorrespondingshortoptions,forusein#!scripts.
Directoriesnamedonthecommandlinenowproduceawarning,notafatalerror,unlessposixor
traditionalareused(seeCommandlinedirectories).
Thegawkinternalswererewritten,bringingthedgawkdebuggerandpossiblyimprovedperformance(see
Debugger).
PertheGNUCodingStandards,dynamicextensionsmustnowdefineaglobalsymbolindicatingthattheyare
GPLcompatible(seePluginLicense).
InPOSIXmode,stringcomparisonsusestrcoll()/wcscoll()(seePOSIXStringComparison).
Theoptionforrawsocketswasremoved,sinceitwasneverimplemented(seeTCP/IPNetworking).
Rangesoftheform[dh]aretreatedasiftheywereintheClocale,nomatterwhatkindofregexpisbeingused,
andevenifposix(seeRangesandLocales).
Supportwasremovedforthefollowingsystems:
Atari
Amiga
BeOS
Cray
MIPSRiscOS
MSDOSwithMicrosoftCompiler
MSWindowswithMicrosoftCompiler
NeXT
SunOS3.x,Sun386(RoadRunner)
Tandem(nonPOSIX)
PrestandardVAXCcompilerforVAX/VMS
Threenewarrays:SYMTAB,FUNCTAB,andPROCINFO["identifiers"](seeAutoset).
Thethreeexecutablesgawk,pgawk,anddgawk,weremergedintoone,namedjustgawk.Asaresultthecommand
lineoptionschanged.
TheDoptioninvokesthedebugger.
Theiandincludeoptionsloadawklibraryfiles.
Thelandloadoptionsloadcompileddynamicextensions.
TheMandbignumoptionsenableMPFR.
Theooptiononlydoesprettyprinting.
Thepoptionisusedforprofiling.
TheRoptionwasremoved.
SupportforhighprecisionarithmeticwithMPFR.(seeArbitraryPrecisionArithmetic).
Theand(),or()andxor()functionschangedtoallowanynumberofarguments,withaminimumoftwo(see
BitwiseFunctions).
Thedynamicextensioninterfacewascompletelyredone(seeDynamicExtensions).
RedirectedgetlinebecameallowedinsideBEGINFILEandENDFILE(seeBEGINFILE/ENDFILE).
Thewherecommandwasaddedtothedebugger(seeExecutionStack).
Next:RangesandLocales,Previous:FeatureHistory,Up:LanguageHistory[Contents][Index]
A.7CommonExtensionsSummary
Thefollowingtablesummarizesthecommonextensionssupportedbygawk,BrianKernighansawk,andmawk,thethree
mostwidelyusedfreelyavailableversionsofawk(seeOtherVersions).
Feature
\xescapesequence
FSasnullstring
BWKawk mawk gawk Nowstandard

X
X X
X
X X
355/479
29/7/2016
TheGNUAwkUsersGuide
/dev/stdinspecialfile
X
/dev/stdoutspecialfile X
/dev/stderrspecialfile X
deletewithoutsubscript X
fflush()function
X
length()ofanarray
X
nextfilestatement
X
**and**=operators
X
funckeyword
X
BINMODEvariable
RSasregexp
Timerelatedfunctions
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
Next:Contributors,Previous:CommonExtensions,Up:LanguageHistory[Contents][Index]
A.8RegexpRangesandLocales:ALongSadStory
Thissectiondescribestheconfusinghistoryofrangeswithinregularexpressionsandtheirinteractionswithlocales,and
howthisaffecteddifferentversionsofgawk.
TheoriginalUnixtoolsthatworkedwithregularexpressionsdefinedcharacterranges(suchas[az])tomatchany
characterbetweenthefirstcharacterintherangeandthelastcharacterintherange,inclusive.Orderingwasbasedon
thenumericvalueofeachcharacterinthemachinesnativecharacterset.Thus,onASCIIbasedsystems,[az]
matchedallthelowercaseletters,andonlythelowercaseletters,asthenumericvaluesforthelettersfromathrough
zwerecontiguous.(OnanEBCDICsystem,therange[az]includesadditionalnonalphabeticcharactersaswell.)
AlmostallintroductoryUnixliteratureexplainedrangeexpressionsasworkinginthisfashion,andinparticular,would
teachthatthecorrectwaytomatchlowercaseletterswaswith[az],andthat[AZ]wasthecorrectwayto
matchuppercaseletters.Andindeed,thiswastrue.107
The1992POSIXstandardintroducedtheideaoflocales(seeLocales).Becausemanylocalesincludeotherletters
besidestheplain26lettersoftheEnglishalphabet,thePOSIXstandardaddedcharacterclasses(seeBracket
Expressions)asawaytomatchdifferentkindsofcharactersbesidesthetraditionalonesintheASCIIcharacterset.
However,thestandardchangedtheinterpretationofrangeexpressions.Inthe"C"and"POSIX"locales,arange
expressionlike[adxz]isstillequivalentto[abcdxyz],asinASCII.Butoutsidethoselocales,theorderingwas
definedtobebasedoncollationorder.
Whatdoesthatmean?Inmanylocales,AandaarebothlessthanB.Inotherwords,theselocalessortcharactersin
dictionaryorder,and[adxz]istypicallynotequivalentto[abcdxyz]instead,itmightbeequivalentto
[ABCXYabcdxyz],forexample.
Thispointneedstobeemphasized:muchliteratureteachesthatyoushoulduse[az]tomatchalowercasecharacter.
ButonsystemswithnonASCIIlocales,thisalsomatchesalloftheuppercasecharactersexceptAorZ!Thiswasa
continuouscauseofconfusion,evenwellintothetwentyfirstcentury.
Todemonstratetheseissues,thefollowingexampleusesthesub()function,whichdoestextreplacement(seeString
Functions).Here,theintentistoremovetrailinguppercasecharacters:
$echosomething1234abc|gawk3.1.8'{sub("[AZ]*$","");print}'
|something1234a
Thisoutputisunexpected,asthebcattheendofsomething1234abcshouldnotnormallymatch[AZ]*.Thisresult
isduetothelocalesetting(andthusyoumaynotseeitonyoursystem).
356/479
29/7/2016
TheGNUAwkUsersGuide
Similarconsiderationsapplytootherranges.Forexample,["/]isperfectlyvalidinASCII,butisnotvalidinmany
Unicodelocales,suchasen_US.UTF8.
Earlyversionsofgawkusedregexpmatchingcodethatwasnotlocaleaware,sorangeshadtheirtraditional
interpretation.
Whengawkswitchedtousinglocaleawareregexpmatchers,theproblemsbeganespeciallyasbothGNU/Linuxand
commercialUnixvendorsstartedimplementingnonASCIIlocales,andmakingthemthedefault.Perhapsthemost
frequentlyaskedquestionbecamesomethinglike,Whydoes[AZ]matchlowercaseletters?!?
Thissituationexistedforcloseto10years,ifnotmore,andthegawkmaintainergrewwearyoftryingtoexplainthat
gawkwasbeingnicelystandardscompliant,andthattheissuewasintheuserslocale.Duringthedevelopmentof
version4.0,hemodifiedgawktoalwaystreatrangesintheoriginal,prePOSIXfashion,unlessposixwasused(see
Options).108
Fortunately,shortlybeforethefinalreleaseofgawk4.0,themaintainerlearnedthatthe2008standardhadchangedthe
definitionofranges,suchthatoutsidethe"C"and"POSIX"locales,themeaningofrangeexpressionswasundefined.109
Byusingthislovelytechnicalterm,thestandardgiveslicensetoimplementorstoimplementrangesinwhateverway
theychoose.ThegawkmaintainerchosetoapplytheprePOSIXmeaningbothwiththedefaultregexpmatchingand
whentraditionalorposixareused.InallcasesgawkremainsPOSIXcompliant.
Next:Historysummary,Previous:RangesandLocales,Up:LanguageHistory[Contents][Index]
A.9MajorContributorstogawk
Alwaysgivecreditwherecreditisdue.
Anonymous
Thissectionnamesthemajorcontributorstogawkand/orthisWebpage,inapproximatechronologicalorder:
Dr.AlfredV.Aho,Dr.PeterJ.Weinberger,andDr.BrianW.Kernighan,allofBellLaboratories,designedand
implementedUnixawk,fromwhichgawkgetsthemajorityofitsfeatureset.
PaulRubindidtheinitialdesignandimplementationin1986,andwrotethefirstdraft(around40pages)ofthis
Webpage.
JayFenlasonfinishedtheinitialimplementation.
DianeCloserevisedthefirstdraftofthisWebpage,bringingittoaround90pages.
RichardStallmanhelpedfinishtheimplementationandtheinitialdraftofthisWebpage.Heisalsothefounder
oftheFSFandtheGNUProject.
JohnWoodscontributedpartsofthecode(mostlyfixes)intheinitialversionofgawk.
In1988,DavidTruemantookoverprimarymaintenanceofgawk,makingitcompatiblewithnewawk,and
greatlyimprovingitsperformance.
ConradKwok,ScottGarfinkle,andKentWilliamsdidtheinitialportstoMSDOSwithvariousversionsof
MSC.
PatRankinprovidedtheVMSportanditsdocumentation.
HalPetersonprovidedhelpinportinggawktoCraysystems.(Thisisnolongersupported.)
KaiUweRommelprovidedtheinitialporttoOS/2anditsdocumentation.
MichalJaegermannprovidedtheporttoAtarisystemsanditsdocumentation.(Thisportisnolongersupported.)
Hecontinuestoprovideportabilitychecking,andhasdonealotofworktomakesuregawkworksonnon32bit
systems.
FredFishprovidedtheporttoAmigasystemsanditsdocumentation.(WithFredssadpassing,thisisnolonger
supported.)
ScottDeifikcurrentlymaintainstheMSDOSportusingDJGPP.
EliZaretskiicurrentlymaintainstheMSWindowsportusingMinGW.
JuanGrigeraprovidedaporttoWindows32systems.(Thisisnolongersupported.)
Formanyyears,Dr.DarrelHankersonactedascoordinatorforthevariousportstodifferentPCplatformsand
357/479
29/7/2016
TheGNUAwkUsersGuide
createdbinarydistributionsforvariousPCoperatingsystems.Hewasalsoinstrumentalinkeepingthe
documentationuptodateforthevariousPCplatforms.
ChristosZoulasprovidedtheextension()builtinfunctionfordynamicallyaddingnewfunctions.(Thiswas
obsoletedatgawk4.1.)
JrgenKahrscontributedtheinitialversionoftheTCP/IPnetworkingcodeanddocumentation,andmotivated
theinclusionofthe|&operator.
StephenDaviesprovidedtheinitialporttoTandemsystemsanditsdocumentation.(However,thisisnolonger
supported.)Hewasalsoinstrumentalintheinitialworktointegratethebytecodeinternalsintothegawkcode
base.
MatthewWoehlkeprovidedimprovementsforTandemsPOSIXcompliantsystems.
MartinBrownprovidedtheporttoBeOSanditsdocumentation.(Thisisnolongersupported.)
ArnoPetersdidtheinitialworktoconvertgawktouseGNUAutomakeandGNUgettext.
AlanJ.Broderprovidedtheinitialversionoftheasort()functionaswellasthecodefortheoptionalthird
argumenttothematch()function.
AndreasBueningupdatedthegawkportforOS/2.
IsamuHasegawa,ofIBMinJapan,contributedsupportformultibytecharacters.
MichaelBenzingercontributedtheinitialcodeforswitchstatements.
PatrickT.J.McPheecontributedthecodefordynamicloadinginWindows32environments.(Thisisnolonger
supported.)
AndersWallinhelpedkeeptheVMSportgoingforseveralyears.
AssafGordoncontributedthecodetoimplementthesandboxoption.
JohnHaquemadethefollowingcontributions:
Themodificationstoconvertgawkintoabytecodeinterpreter,includingthedebugger
Theadditionoftruearraysofarrays
Theadditionalmodificationsforsupportofarbitraryprecisionarithmetic
TheinitialtextofArbitraryPrecisionArithmetic
Theworktomergethethreeversionsofgawkintoone,forthe4.1release
Improvedarrayinternalsforarraysindexedbyintegers
TheimprovedarraysortingfeatureswerealsodrivenbyJohn,togetherwithPatRankin
PanosPapadopouloscontributedtheoriginaltextforIncludeFiles.
EfraimYawitzcontributedtheoriginaltextforDebugger.
ThedevelopmentoftheextensionAPIfirstreleasedwithgawk4.1wasdrivenprimarilybyArnoldRobbinsand
AndrewSchorr,withnotablecontributionsfromtherestofthedevelopmentteam.
JohnMalmbergcontributedsignificantimprovementstotheOpenVMSportandtherelateddocumentation.
AntonioGiovanniColomborewroteanumberofexamplesintheearlychaptersthatwereseverelydated,for
whichIamincrediblygrateful.
ArnoldRobbinshasbeenworkingongawksince1988,atfirsthelpingDavidTrueman,andastheprimary
maintainersincearound1994.
Previous:Contributors,Up:LanguageHistory[Contents][Index]
A.10Summary
Theawklanguagehasevolvedovertime.ThefirstreleasewaswithV7Unix,circa1978.In1987,forSystemV
Release3.1,majoradditions,includinguserdefinedfunctions,weremadetothelanguage.Additionalchanges
weremadeforSystemVRelease4,in1989.Sincethen,furtherminorchangeshavehappenedundertheauspices
ofthePOSIXstandard.
BrianKernighansawkprovidesasmallnumberofextensionsthatareimplementedincommonwithother
versionsofawk.
gawkprovidesalargenumberofextensionsoverPOSIXawk.Theycanbedisabledwitheitherthetraditional
orposixoptions.
TheinteractionofPOSIXlocalesandregexpmatchingingawkhasbeenconfusingovertheyears.Today,gawk
implementsRationalRangeInterpretation,whererangesoftheform[az]matchonlythecharacters
numericallybetweenathroughzinthemachinesnativecharacterset.UsuallythisisASCII,butitcanbe
EBCDIConIBMS/390systems.
Manypeoplehavecontributedtogawkdevelopmentovertheyears.Wehopethatthelistprovidedinthischapter
358/479
29/7/2016
TheGNUAwkUsersGuide
iscompleteandgivestheappropriatecreditwherecreditisdue.
Next:Notes,Previous:LanguageHistory,Up:Top[Contents][Index]
Thisappendixprovidesinstructionsforinstallinggawkonthevariousplatformsthataresupportedbythedevelopers.
TheprimarydevelopersupportsGNU/Linux(andUnix),whereastheotherportsarecontributed.SeeBugs,forthe
emailaddressesofthepeoplewhomaintaintherespectiveports.
GawkDistribution: Whatisinthegawkdistribution.
UnixInstallation:
InstallinggawkundervariousversionsofUnix.
NonUnixInstallation: InstallationonOtherOperatingSystems.
Bugs:
ReportingProblemsandBugs.
OtherVersions:
Otherfreelyavailableawkimplementations.
Installationsummary: Summaryofinstallation.
Next:UnixInstallation,Up:Installation[Contents][Index]
B.1ThegawkDistribution
Thissectiondescribeshowtogetthegawkdistribution,howtoextractit,andthenwhatisinthevariousfilesand
subdirectories.
Getting:
Howtogetthedistribution.
Extracting:
Howtoextractthedistribution.
Distributioncontents: Whatisinthedistribution.
Next:Extracting,Up:GawkDistribution[Contents][Index]
B.1.1GettingthegawkDistribution
TherearetwowaystogetGNUsoftware:
Copyitfromsomeoneelsewhoalreadyhasit.
RetrievegawkfromtheInternethostftp.gnu.org,inthedirectory/gnu/gawk.Bothanonymousftpandhttp
accessaresupported.Ifyouhavethewgetprogram,youcanuseacommandlikethefollowing:
wgethttp://ftp.gnu.org/gnu/gawk/gawk4.1.2.tar.gz
TheGNUsoftwarearchiveismirroredaroundtheworld.Theuptodatelistofmirrorsitesisavailablefromthemain
FSFwebsite.Trytouseoneofthemirrorstheywillbelessbusy,andyoucanusuallyfindoneclosertoyoursite.
Next:Distributioncontents,Previous:Getting,Up:GawkDistribution[Contents][Index]
B.1.2ExtractingtheDistribution
gawkisdistributedasseveraltarfilescompressedwithdifferentcompressionprograms:gzip,bzip2,andxz.For
simplicity,therestoftheseinstructionsassumeyouareusingtheonecompressedwiththeGNUGzipprogram(gzip).
Onceyouhavethedistribution(e.g.,gawk4.1.2.tar.gz),usegziptoexpandthefileandthenusetartoextractit.You
canusethefollowingpipelinetoproducethegawkdistribution:
359/479
29/7/2016
TheGNUAwkUsersGuide
gzipdcgawk4.1.2.tar.gz|tarxvpf
OnasystemwithGNUtar,youcanlettardothedecompressionforyou:
tarxvpzfgawk4.1.2.tar.gz
Extractingthearchivecreatesadirectorynamedgawk4.1.2inthecurrentdirectory.
ThedistributionfilenameisoftheformgawkV.R.P.tar.gz.TheVrepresentsthemajorversionofgawk,theR
representsthecurrentreleaseofversionV,andthePrepresentsapatchlevel,meaningthatminorbugshavebeenfixed
intherelease.Thecurrentpatchlevelis2,butwhenretrievingdistributions,youshouldgettheversionwiththehighest
version,release,andpatchlevel.(Note,however,thatpatchlevelsgreaterthanorequalto70denotebetaor
nonproductionsoftwareyoumightnotwanttoretrievesuchaversionunlessyoudontmindexperimenting.)Ifyouare
notonaUnixorGNU/Linuxsystem,youneedtomakeotherarrangementsforgettingandextractingthegawk
distribution.Youshouldconsultalocalexpert.
Previous:Extracting,Up:GawkDistribution[Contents][Index]
B.1.3ContentsofthegawkDistribution
ThegawkdistributionhasanumberofCsourcefiles,documentationfiles,subdirectories,andfilesrelatedtothe
configurationprocess(seeUnixInstallation),aswellasseveralsubdirectoriesrelatedtodifferentnonUnixoperating
systems:
Various.c,.y,and.hfiles
Thesefilescontaintheactualgawksourcecode.
ABOUTNLS
AfilecontaininginformationaboutGNUgettextandtranslations.
AUTHORS
Afilewithsomeinformationabouttheauthorshipofgawk.ItexistsonlytosatisfythepedantsattheFree
SoftwareFoundation.
README
README_d/README.*
Descriptivefiles:READMEforgawkunderUnixandtherestforthevarioushardwareandsoftwarecombinations.
INSTALL
Afileprovidinganoverviewoftheconfigurationandinstallationprocess.
ChangeLog
Adetailedlistofsourcecodechangesasbugsarefixedorimprovementsmade.
ChangeLog.0
Anolderlistofsourcecodechanges.
NEWS
Alistofchangestogawksincethelastreleaseorpatch.
NEWS.0
360/479
29/7/2016
TheGNUAwkUsersGuide
Anolderlistofchangestogawk.
COPYING
TheGNUGeneralPublicLicense.
POSIX.STD
AdescriptionofbehaviorsinthePOSIXstandardforawkthatareleftundefined,orwheregawkmaynotcomply
fully,aswellasalistofthingsthatthePOSIXstandardshoulddescribebutdoesnot.
doc/awkforai.txt
Pointerstotheoriginaldraftofashortarticledescribingwhygawkisagoodlanguageforartificialintelligence
(AI)programming.
doc/bc_notes
Abriefdescriptionofgawksbytecodeinternals.
doc/README.card
doc/ad.block
doc/awkcard.in
doc/cardfonts
doc/colors
doc/macros
doc/no.colors
doc/setter.outline
Thetroffsourceforafivecolorawkreferencecard.AmodernversionoftroffsuchasGNUtroff(groff)is
neededtoproducethecolorversion.SeethefileREADME.cardforinstructionsifyouhaveanoldertroff.
doc/gawk.1
Thetroffsourceforamanualpagedescribinggawk.ThisisdistributedfortheconvenienceofUnixusers.
doc/gawktexi.in
doc/sidebar.awk
TheTexinfosourcefileforthisWebpage.Itshouldbeprocessedbydoc/sidebar.awkbeforeprocessingwith
texi2dviortexi2pdftoproduceaprinteddocument,andwithmakeinfotoproduceanInfoorHTMLfile.The
Makefiletakescareofthisprocessingandproducesprintableoutputviatexi2dviortexi2pdf.
doc/gawk.texi
Thefileproducedafterprocessinggawktexi.inwithsidebar.awk.
doc/gawk.info
ThegeneratedInfofileforthisWebpage.
doc/gawkinet.texi
TheTexinfosourcefileforTCP/IPInternetworkingwithgawk.ItshouldbeprocessedwithTeX(viatexi2dvior
texi2pdf)toproduceaprinteddocumentandwithmakeinfotoproduceanInfoorHTMLfile.
doc/gawkinet.info
ThegeneratedInfofileforTCP/IPInternetworkingwithgawk.
doc/igawk.1
ThetroffsourceforamanualpagedescribingtheigawkprogrampresentedinIgawkProgram.
361/479
29/7/2016
TheGNUAwkUsersGuide
doc/Makefile.in
TheinputfileusedduringtheconfigurationprocesstogeneratetheactualMakefileforcreatingthe
documentation.
Makefile.am
*/Makefile.am
FilesusedbytheGNUAutomakesoftwareforgeneratingtheMakefile.infilesusedbyAutoconfandconfigure.
Makefile.in
aclocal.m4
bisonfix.awk
config.guess
configh.in
configure.ac
configure
custom.h
depcomp
installsh
missing_d/*
mkinstalldirs
m4/*
ThesefilesandsubdirectoriesareusedwhenconfiguringandcompilinggawkforvariousUnixsystems.Mostof
themareexplainedinUnixInstallation.Therestaretheretosupportthemaininfrastructure.
po/*
Thepolibrarycontainsmessagetranslations.
awklib/extract.awk
awklib/Makefile.am
awklib/Makefile.in
awklib/eg/*
Theawklibdirectorycontainsacopyofextract.awk(seeExtractProgram),whichcanbeusedtoextractthe
sampleprogramsfromtheTexinfosourcefileforthisWebpage.ItalsocontainsaMakefile.infile,which
configureusestogenerateaMakefile.Makefile.amisusedbyGNUAutomaketocreateMakefile.in.Thelibrary
functionsfromLibraryFunctions,andtheigawkprogramfromIgawkProgram,areincludedasreadytousefiles
inthegawkdistribution.Theyareinstalledaspartoftheinstallationprocess.TherestoftheprogramsinthisWeb
pageareavailableinappropriatesubdirectoriesofawklib/eg.
extension/*
Thesourcecode,manualpages,andinfrastructurefilesforthesampleextensionsincludedwithgawk.See
DynamicExtensions,formoreinformation.
posix/*
FilesneededforbuildinggawkonPOSIXcompliantsystems.
pc/*
FilesneededforbuildinggawkunderMSWindowsandOS/2(seePCInstallation,fordetails).
vms/*
FilesneededforbuildinggawkunderVax/VMSandOpenVMS(seeVMSInstallation,fordetails).
test/*
Atestsuiteforgawk.Youcanusemakecheckfromthetoplevelgawkdirectorytorunyourversionofgawk
againstthetestsuite.Ifgawksuccessfullypassesmakecheck,thenyoucanbeconfidentofasuccessfulport.
362/479
29/7/2016
TheGNUAwkUsersGuide
Next:NonUnixInstallation,Previous:GawkDistribution,Up:Installation[Contents][Index]
B.2CompilingandInstallinggawkonUnixLikeSystems
Usually,youcancompileandinstallgawkbytypingonlytwocommands.However,ifyouuseanunusualsystem,you
mayneedtoconfiguregawkforyoursystemyourself.
QuickInstallation:
CompilinggawkunderUnix.
AdditionalConfigurationOptions: Othercompiletimeoptions.
ConfigurationPhilosophy:
Howitsallsupposedtowork.
Next:AdditionalConfigurationOptions,Up:UnixInstallation[Contents][Index]
B.2.1CompilinggawkforUnixLikeSystems
ThenormalinstallationstepsshouldworkonallmoderncommercialUnixderivedsystems,GNU/Linux,BSDbased
systems,andtheCygwinenvironmentforMSWindows.
Afteryouhaveextractedthegawkdistribution,cdtogawk4.1.2.AswithmostGNUsoftware,youconfiguregawkfor
yoursystembyrunningtheconfigureprogram.ThisprogramisaBourneshellscriptthatisgeneratedautomatically
usingGNUAutoconf.(TheAutoconfsoftwareisdescribedfullyinAutoconfGeneratingAutomaticConfiguration
Scripts,whichcanbefoundonlineattheFreeSoftwareFoundationswebsite.)
Toconfiguregawk,simplyrunconfigure:
sh./configure
ThisproducesaMakefileandconfig.htailoredtoyoursystem.Theconfig.hfiledescribesvariousfactsaboutyour
system.YoumightwanttoedittheMakefiletochangetheCFLAGSvariable,whichcontrolsthecommandlineoptions
thatarepassedtotheCcompiler(suchasoptimizationlevelsorcompilingfordebugging).
Alternatively,youcanaddyourownvaluesformostmakevariablesonthecommandline,suchasCCandCFLAGS,when
runningconfigure:
CC=ccCFLAGS=gsh./configure
SeethefileINSTALLinthegawkdistributionforallthedetails.
AfteryouhaverunconfigureandpossiblyeditedtheMakefile,type:
make
Shortlythereafter,youshouldhaveanexecutableversionofgawk.Thatsallthereistoit!Toverifythatgawkisworking
properly,runmakecheck.Allofthetestsshouldsucceed.Ifthesestepsdonotwork,orifanyofthetestsfail,check
thefilesintheREADME_ddirectorytoseeifyouvefoundaknownproblem.Ifthefailureisnotdescribedthere,sendina
bugreport(seeBugs).
Ofcourse,onceyouvebuiltgawk,itislikelythatyouwillwishtoinstallit.Todoso,youneedtorunthecommand
makeinstall,asauserwiththeappropriatepermissions.Howtodothisvariesbysystem,butonmanysystemsyou
canusethesudocommandtodoso.Thecommandthenbecomessudomakeinstall.Itislikelythatyouwillbeasked
foryourpassword,andyouwillhavetohavebeensetuppreviouslyasauserwhoisallowedtorunthesudocommand.
Next:ConfigurationPhilosophy,Previous:QuickInstallation,Up:UnixInstallation[Contents][Index]
B.2.2AdditionalConfigurationOptions
363/479
29/7/2016
TheGNUAwkUsersGuide
Thereareseveraladditionaloptionsyoumayuseontheconfigurecommandlinewhencompilinggawkfromscratch,
including:
disableextensions
Disableconfiguringandbuildingthesampleextensionsintheextensiondirectory.Thisisusefulforcross
compiling.Thedefaultactionistodynamicallycheckiftheextensionscanbeconfiguredandcompiled.
disablelint
Disablealllintcheckingwithingawk.Thelintandlintoldoptions(seeOptions)areaccepted,butsilently
donothing.Similarly,settingtheLINTvariable(seeUsermodified)hasnoeffectontherunningawkprogram.
WhenusedwiththeGNUCompilerCollections(GCCs)automaticdeadcodeelimination,thisoptioncuts
almost23KbytesoffthesizeofthegawkexecutableonGNU/Linuxx86_64systems.Resultsonothersystems
andwithothercompilersarelikelytovary.Usingthisoptionmaybringyousomeslightperformance
improvement.
CAUTION:Usingthisoptionwillcausesomeofthetestsinthetestsuitetofail.Thisoptionmaybe
removedatalaterdate.
disablenls
Disableallmessagetranslationfacilities.Thisisusuallynotdesirable,butitmaybringyousomeslight
performanceimprovement.
withwhinyuserstrftime
ForceuseoftheincludedversionoftheCstrftime()functionfordeficientsystems.
Usethecommand./configurehelptoseethefulllistofoptionssuppliedbyconfigure.
Previous:AdditionalConfigurationOptions,Up:UnixInstallation[Contents][Index]
B.2.3TheConfigurationProcess
ThissectionisofinterestonlyifyouknowsomethingaboutusingtheClanguageandUnixlikeoperatingsystems.
Thesourcecodeforgawkgenerallyattemptstoadheretoformalstandardswhereverpossible.Thismeansthatgawkuses
libraryroutinesthatarespecifiedbytheISOCstandardandbythePOSIXoperatingsysteminterfacestandard.The
gawksourcecoderequiresusinganISOCcompiler(the1990standard).
ManyUnixsystemsdonotsupportallofeithertheISOorthePOSIXstandards.Themissing_dsubdirectoryinthegawk
distributioncontainsreplacementversionsofthosefunctionsthataremostlikelytobemissing.
Theconfig.hfilethatconfigurecreatescontainsdefinitionsthatdescribefeaturesoftheparticularoperatingsystem
whereyouareattemptingtocompilegawk.Thethreethingsdescribedbythisfileare:whatheaderfilesareavailable,so
thattheycanbecorrectlyincluded,what(supposedly)standardfunctionsareactuallyavailableinyourClibraries,and
variousmiscellaneousfactsaboutyouroperatingsystem.Forexample,theremaynotbeanst_blksizeelementinthe
statstructure.Inthiscase,HAVE_STRUCT_STAT_ST_BLKSIZEisundefined.
ItispossibleforyourCcompilertolietoconfigure.Itmaydosobynotexitingwithanerrorwhenalibraryfunctionis
notavailable.Togetaroundthis,editthecustom.hfile.Usean#ifdefthatisappropriateforyoursystem,andeither
#defineanyconstantsthatconfigureshouldhavedefinedbutdidnt,or#undefanyconstantsthatconfiguredefinedand
shouldnothave.Thecustom.hfileisautomaticallyincludedbytheconfig.hfile.
ItisalsopossiblethattheconfigureprogramgeneratedbyAutoconfwillnotworkonyoursysteminsomeother
fashion.Ifyoudohaveaproblem,theconfigure.acfileistheinputforAutoconf.Youmaybeabletochangethisfile
andgenerateanewversionofconfigurethatworksonyoursystem(seeBugs,forinformationonhowtoreport
364/479
29/7/2016
TheGNUAwkUsersGuide
problemsinconfiguringgawk).Thesamemechanismmaybeusedtosendinupdatestoconfigure.acand/orcustom.h.
Next:Bugs,Previous:UnixInstallation,Up:Installation[Contents][Index]
B.3InstallationonOtherOperatingSystems
ThissectiondescribeshowtoinstallgawkonvariousnonUnixsystems.
PCInstallation: InstallingandCompilinggawkonMSDOSandOS/2.
VMSInstallation: InstallinggawkonVMS.
Next:VMSInstallation,Up:NonUnixInstallation[Contents][Index]
B.3.1InstallationonPCOperatingSystems
ThissectioncoversinstallationandusageofgawkonIntelarchitecturemachinesrunningMSDOS,anyversionofMS
Windows,orOS/2.Inthissection,thetermWindows32referstoanyofMicrosoftWindows
95/98/ME/NT/2000/XP/Vista/7/8.
ThelimitationsofMSDOS(andMSDOSshellsundertheotheroperatingsystems)havemeantthatvariousDOS
extendersareoftenusedwithprogramssuchasgawk.ThevaryingcapabilitiesofMicrosoftWindows3.1and
Windows32canaddtotheconfusion.Foranoverviewoftheconsiderations,refertoREADME_d/README.pcinthe
distribution.
PCBinaryInstallation: Installingaprepareddistribution.
PCCompiling:
CompilinggawkforMSDOS,Windows32,andOS/2.
PCTesting:
TestinggawkonPCsystems.
PCUsing:
RunninggawkonMSDOS,Windows32andOS/2.
Cygwin:
BuildingandrunninggawkforCygwin.
MSYS:
UsinggawkInTheMSYSEnvironment.
Next:PCCompiling,Up:PCInstallation[Contents][Index]
B.3.1.1InstallingaPreparedDistributionforPCSystems
IfyouhavereceivedabinarydistributionpreparedbytheMSDOSmaintainers,thengawkandthenecessarysupport
filesappearunderthegnudirectory,withexecutablesingnu/bin,librariesingnu/lib/awk,andmanualpagesunder
gnu/man.Thisisdesignedforeasyinstallationtoa/gnudirectoryonyourdrivehowever,thefilescanbeinstalled
anywhereprovidedAWKPATHissetproperly.Regardlessoftheinstallationdirectory,thefirstlineofigawk.cmdand
igawk.bat(ingnu/bin)mayneedtobeedited.
Thebinarydistributioncontainsaseparatefiledescribingthecontents.Inparticular,itmayincludemorethanone
versionofthegawkexecutable.
OS/2(32bit,EMX)binarydistributionsarepreparedforthe/usrdirectoryofyourpreferreddrive.SetUNIXROOTtoyour
installationdrive(e.g.,e:)ifyouwanttoinstallgawkontoanotherdrivethanthehardcodeddefaultc:.Executables
appearin/usr/bin,librariesunder/usr/share/awk,manualpagesunder/usr/man,Texinfodocumentationunder
/usr/info,andNLSfilesunder/usr/share/locale.NotethatthefilescanbeinstalledanywhereprovidedAWKPATHisset
properly.
Ifyoualreadyhaveafile/usr/info/dirfromanotherpackagedonotoverwriteit!Insteadenterthefollowing
commandsatyourprompt(replacex:byyourinstallationdrive):
installinfoinfodir=x:/usr/infox:/usr/info/gawk.info
installinfoinfodir=x:/usr/infox:/usr/info/gawkinet.info
365/479
29/7/2016
TheGNUAwkUsersGuide
Thebinarydistributionmaycontainaseparatefilecontainingadditionalormoredetailedinstallationinstructions.
Next:PCTesting,Previous:PCBinaryInstallation,Up:PCInstallation[Contents][Index]
B.3.1.2CompilinggawkforPCOperatingSystems
gawkcanbecompiledforMSDOS,Windows32,andOS/2usingtheGNUdevelopmenttoolsfromDJDelorie(DJGPP:
MSDOSonly),MinGW(Windows32)orEberhardMattes(EMX:MSDOS,Windows32andOS/2).Thefile
README_d/README.pcinthegawkdistributioncontainsadditionalnotes,andpc/Makefilecontainsimportantinformation
oncompilationoptions.
TobuildgawkforMSDOSandWindows32,copythefilesinthepcdirectory(exceptforChangeLog)tothedirectory
withtherestofthegawksources,theninvokemakewiththeappropriatetargetnameasanargumenttobuildgawk.The
Makefilecopiedfromthepcdirectorycontainsaconfigurationsectionwithcommentsandmayneedtobeeditedin
ordertoworkwithyourmakeutility.
TheMakefilesupportsanumberoftargetsforbuildingvariousMSDOSandWindows32versions.Alistoftargetsis
printedifthemakecommandisgivenwithoutatarget.Asanexample,tobuildgawkusingtheDJGPPtools,entermake
djgpp.(TheDJGPPtoolsneededforthebuildmaybefoundatftp://ftp.delorie.com/pub/djgpp/current/v2gnu/.)To
buildanativeMSWindowsbinaryofgawkusingtheMinGWtools,typemakemingw32.
The32bitEMXversionofgawkworksoutoftheboxunderOS/2.However,itishighlyrecommendedtouseGCC
2.95.3forthecompilation.Inprinciple,itispossibletocompilegawkthefollowingway:
$./configure
$make
Thisisnotrecommended,though.TogetanOMFexecutableyoushouldusethefollowingcommandsatyoursh
prompt:
$CFLAGS="O2ZomfZmt"
$exportCFLAGS
$LDFLAGS="sZcrtdllZlinker/exepack:2Zlinker/pm:vioZstack0x6000"
$exportLDFLAGS
$RANLIB="echo"
$exportRANLIB
$./configureprefix=c:/usr
$makeAR=emxomfar
ThesearejustsuggestionsforusewithGCC2.x.Youmayuseanyothersetof(selfconsistent)environmentvariables
andcompilerflags.
IfyouuseGCC2.95itisrecommendedtousealso:
$LIBS="lgcc"
$exportLIBS
Youcanalsogetana.outexecutableifyouprefer:
$CFLAGS="O2Zmt"
$exportCFLAGS
$LDFLAGS="sZstack0x6000"
$LIBS="lgcc"
$unsetRANLIB
$./configureprefix=c:/usr
$make
NOTE:Compilationofa.outexecutablesalsoworkswithGCC3.2.VersionslaterthanGCC3.2havenot
beentestedsuccessfully.
makeinstallworksasexpectedwiththeEMXbuild.
366/479
29/7/2016
TheGNUAwkUsersGuide
NOTE:AncientOS/2portsofGNUmakearenotabletohandletheMakefilesofthispackage.Ifyou
encounteranyproblemswithmake,tryGNUMake3.79.1orlaterversions.Youshouldfindthelatest
versiononftp://hobbes.nmsu.edu/pub/os2/.110
Next:PCUsing,Previous:PCCompiling,Up:PCInstallation[Contents][Index]
B.3.1.3TestinggawkonPCOperatingSystems
UsingmaketorunthestandardtestsandtoinstallgawkrequiresadditionalUnixliketools,includingsh,sed,andcp.In
ordertorunthetests,thetest/*.okfilesmayneedtobeconvertedsothattheyhavetheusualMSDOSstyleendof
linemarkers.Alternatively,runmakecheckCMP="diffa"touseGNUdiffintextmodeinsteadofcmptocomparethe
resultingfiles.
MostofthetestsworkproperlywithStewartsonsshellalongwiththecompanionutilitiesorappropriateGNUutilities.
However,someeditingoftest/Makefileisrequired.Itisrecommendedthatyoucopythefilepc/Makefile.tstoverthe
filetest/Makefileasareplacement.DetailscanbefoundinREADME_d/README.pcandinthefilepc/Makefile.tst.
OnOS/2thepidtestfailsbecausespawnl()isusedinsteadoffork()/execl()tostartchildprocesses.Alsothembfw1
andmbprintf1testsfailbecausetheneededmultibytefunctionalityisnotavailable.
Next:Cygwin,Previous:PCTesting,Up:PCInstallation[Contents][Index]
B.3.1.4UsinggawkonPCOperatingSystems
UnderMSDOSandMSWindows,theCygwinandMinGWenvironmentssupportboththe|&operatorandTCP/IP
networking(seeTCP/IPNetworking).EMX(OS/2only)supportsatleastthe|&operator.
TheMSDOSandMSWindowsversionsofgawksearchforprogramfilesasdescribedinAWKPATHVariable.
However,semicolons(ratherthancolons)separateelementsintheAWKPATHvariable.IfAWKPATHisnotsetorisempty,
thenthedefaultsearchpathis.;c:/lib/awk;c:/gnu/lib/awk.
ThesearchpathforOS/2(32bit,EMX)isdeterminedbytheprefixdirectory(mostlikely/usrorc:/usr)thathasbeen
specifiedasanoptionoftheconfigurescriptasisthecasefortheUnixversions.Ifc:/usristheprefixdirectorythen
thedefaultsearchpathcontains.andc:/usr/share/awk.Additionally,tosupportbinarydistributionsofgawkforOS/2
systemswhosedrivec:mightnotsupportlongfilenamesormightnotexistatall,thereisaspecialenvironment
variable.IfUNIXROOTspecifiesadrivethenthisspecificdriveisalsosearchedforprogramfiles.E.g.,ifUNIXROOTissetto
e:thecompletedefaultsearchpathis.;c:/usr/share/awk;e:/usr/share/awk.
Anshlikeshell(asopposedtocommand.comunderMSDOSorcmd.exeunderMSWindowsorOS/2)maybeusefulfor
awkprogramming.TheDJGPPcollectionoftoolsincludesanMSDOSportofBash,andseveralshellsareavailablefor
OS/2,includingksh.
UnderMSWindows,OS/2andMSDOS,gawk(andmanyothertextprograms)silentlytranslatesendofline\r\nto
\noninputand\nto \r\nonoutput.AspecialBINMODEvariable(c.e.)allowscontroloverthesetranslationsandis
interpretedasfollows:
IfBINMODEis"r"orone,thenbinarymodeissetonread(i.e.,notranslationsonreads).
IfBINMODEis"w"ortwo,thenbinarymodeissetonwrite(i.e.,notranslationsonwrites).
IfBINMODEis"rw"or"wr"orthree,binarymodeissetforbothreadandwrite.
BINMODE=nonnullstringisthesameasBINMODE=3(i.e.,notranslationsonreadsorwrites).However,gawk
issuesawarningmessageifthestringisnotoneof"rw"or"wr".
Themodesforstandardinputandstandardoutputaresetonetimeonly(afterthecommandlineisread,butbefore
processinganyoftheawkprogram).SettingBINMODEforstandardinputorstandardoutputisaccomplishedbyusingan
appropriatevBINMODE=Noptiononthecommandline.BINMODEissetatthetimeafileorpipeisopenedandcannotbe
changedmidstream.
367/479
29/7/2016
TheGNUAwkUsersGuide
ThenameBINMODEwaschosentomatchmawk(seeOtherVersions).mawkandgawkhandleBINMODEsimilarlyhowever,
mawkaddsaWBINMODE=NoptionandanenvironmentvariablethatcansetBINMODE,RS,andORS.Thefilesbinmode[1
3].awk(undergnu/lib/awkinsomeofthepreparedbinarydistributions)havebeenchosentomatchmawksW
BINMODE=Noption.Thesecanbechangedordiscardedinparticular,thesettingofRSgivingthefewestsurprisesis
opentodebate.mawkusesRS="\r\n"ifbinarymodeissetonread,whichisappropriateforfileswiththeMSDOS
styleendofline.
Toillustrate,thefollowingexamplessetbinarymodeonwritesforstandardoutputandotherfiles,andsetORSasthe
usualMSDOSstyleendofline:
gawkvBINMODE=2vORS="\r\n"
or:
gawkvBINMODE=wfbinmode2.awk
ThesegivethesameresultastheWBINMODE=2optioninmawk.Thefollowingchangestherecordseparatorto"\r\n"
andsetsbinarymodeonreads,butdoesnotaffectthemodeonstandardinput:
gawkvRS="\r\n"e"BEGIN{BINMODE=1}"
or:
gawkfbinmode1.awk
Withproperquoting,inthefirstexamplethesettingofRScanbemovedintotheBEGINrule.
Next:MSYS,Previous:PCUsing,Up:PCInstallation[Contents][Index]
B.3.1.5UsinggawkInTheCygwinEnvironment
gawkcanbebuiltandusedoutoftheboxunderMSWindowsifyouareusingtheCygwinenvironment.This
environmentprovidesanexcellentsimulationofGNU/Linux,usingBash,GCC,GNUMake,andotherGNUprograms.
CompilationandinstallationforCygwinisthesameasforaUnixsystem:
cdgawk4.1.2
./configure
make&&makecheck
WhencomparedtoGNU/Linuxonthesamesystem,theconfiguresteponCygwintakesconsiderablylonger.
However,itdoesfinish,andthenthemakeproceedsasusual.
Previous:Cygwin,Up:PCInstallation[Contents][Index]
B.3.1.6UsinggawkInTheMSYSEnvironment
IntheMSYSenvironmentunderMSWindows,gawkautomaticallyusesbinarymodeforreadingandwritingfiles.
Thus,thereisnoneedtousetheBINMODEvariable.
ThiscancauseproblemswithotherUnixlikecomponentsthathavebeenportedtoMSWindowsthatexpectgawktodo
automatictranslationof"\r\n",becauseitwont.
Previous:PCInstallation,Up:NonUnixInstallation[Contents][Index]
B.3.2CompilingandInstallinggawkonVax/VMSandOpenVMS
368/479
29/7/2016
TheGNUAwkUsersGuide
ThissubsectiondescribeshowtocompileandinstallgawkunderVMS.TheolderdesignationVMSisusedthroughout
torefertoOpenVMS.
VMSCompilation:
HowtocompilegawkunderVMS.
VMSDynamicExtensions: CompilinggawkdynamicextensionsonVMS.
VMSInstallationDetails: HowtoinstallgawkunderVMS.
VMSRunning:
HowtorungawkunderVMS.
VMSGNV:
TheVMSGNVProject.
VMSOldGawk:
AnoldversioncomeswithsomeVMSsystems.
Next:VMSDynamicExtensions,Up:VMSInstallation[Contents][Index]
B.3.2.1CompilinggawkonVMS
TocompilegawkunderVMS,thereisaDCLcommandprocedurethatissuesallthenecessaryCCandLINKcommands.
ThereisalsoaMakefileforusewiththeMMSandMMKutilities.Fromthesourcedirectory,useeither:
$@[.vms]vmsbuild.com
or:
$MMS/DESCRIPTION=[.vms]descrip.mmsgawk
or:
$MMK/DESCRIPTION=[.vms]descrip.mmsgawk
MMKisanopensource,free,nearcloneofMMSandcanbetterhandleODS5volumeswithupperandlowercasefile
names.MMKisavailablefromhttps://github.com/endlesssoftware/mmk.
WithODS5volumesandextendedparsingenabled,thecaseofthetargetparametermayneedtobeexact.
gawkhasbeentestedunderVAX/VMS7.3andAlpha/VMS7.31usingCompaqCV6.4,andunderAlpha/VMS7.3,
Alpha/VMS7.32,andIA64/VMS8.3.ThemostrecentbuildsusedHPCV7.3onAlphaVMS8.3andbothAlphaand
IA64VMS8.4usedHPC7.3.111
SeeVMSGNV,forinformationonbuildinggawkasaPCSIkitthatiscompatiblewiththeGNVproduct.
Next:VMSInstallationDetails,Previous:VMSCompilation,Up:VMSInstallation[Contents][Index]
B.3.2.2CompilinggawkDynamicExtensionsonVMS
TheextensionsthathavebeenportedtoVMScanbebuiltusingoneofthefollowingcommands:
$MMS/DESCRIPTION=[.vms]descrip.mmsextensions
or:
$MMK/DESCRIPTION=[.vms]descrip.mmsextensions
gawkusesAWKLIBPATHaseitheranenvironmentvariableoralogicalnametofindthedynamicextensions.
Dynamicextensionsneedtobecompiledwiththesamecompileroptionsforfloatingpoint,pointersize,andsymbol
namehandlingaswereusedtocompilegawkitself.AlphaandItaniumshoulduseIEEEfloatingpoint.Thepointersize
is32bits,andthesymbolnamehandlingshouldbeexactcasewithCRCshorteningforsymbolslongerthan32bits.
ForAlphaandItanium:
369/479
29/7/2016
TheGNUAwkUsersGuide
/name=(as_is,short)
/float=ieee/ieee_mode=denorm_results
ForVAX:
/name=(as_is,short)
CompiletimemacrosneedtobedefinedbeforethefirstVMSsuppliedheaderfileisincluded,asfollows:
#if(__CRTL_VER>=70200000)&&!defined(__VAX)
#define_LARGEFILE1
#endif
#ifndef__VAX
#ifdef__CRTL_VER
#if__CRTL_VER>=80200000
#define_USE_STD_STAT1
#endif
#endif
#endif
IfyouarewritingyourownextensionstorunonVMS,youmustsupplythesedefinitionsyourself.Theconfig.hfile
createdwhenbuildinggawkonVMSdoesthisforyouifinsteadyouusethatfileorasimilarone,thenyoumust
remembertoincludeitbeforeanyVMSsuppliedheaderfiles.
Next:VMSRunning,Previous:VMSDynamicExtensions,Up:VMSInstallation[Contents][Index]
B.3.2.3InstallinggawkonVMS
Tousegawk,allyouneedisaforeigncommand,whichisaDCLsymbolwhosevaluebeginswithadollarsign.For
example:
$GAWK:==$disk1:[gnubin]gawk
Substitutetheactuallocationofgawk.exefor$disk1:[gnubin].Thesymbolshouldbeplacedinthelogin.comofany
userwhowantstorungawk,sothatitisdefinedeverytimetheuserlogson.Alternatively,thesymbolmaybeplacedin
thesystemwidesylogin.comprocedure,whichallowsalluserstorungawk.
IfyourgawkwasinstalledbyaPCSIkitintotheGNV$GNU:directorytree,theprogramwillbeknownasGNV$GNU:
[bin]gnv$gawk.exeandthehelpfilewillbeGNV$GNU:[vms_help]gawk.hlp.
ThePCSIkitalsoinstallsaGNV$GNU:[vms_bin]gawk_verb.cldfilethatcanbeusedtoaddgawkandawkasDCL
commands.
Forjustthecurrentprocessyoucanuse:
$setcommandgnv$gnu:[vms_bin]gawk_verb.cld
OrthesystemmanagercanuseGNV$GNU:[vms_bin]gawk_verb.cldtoaddthegawkandawktothesystemwide
DCLTABLES.
TheDCLsyntaxisdocumentedinthegawk.hlpfile.
Optionally,thegawk.hlpentrycanbeloadedintoaVMShelplibrary:
$LIBRARY/HELPsys$help:helplib[.vms]gawk.hlp
(YoumaywanttosubstituteasitespecifichelplibraryratherthanthestandardVMSlibraryHELPLIB.)Afterloading
thehelptext,thecommand:
$HELPGAWK
370/479
29/7/2016
TheGNUAwkUsersGuide
providesinformationaboutboththegawkimplementationandtheawkprogramminglanguage.
ThelogicalnameAWK_LIBRARYcandesignateadefaultlocationforawkprogramfiles.Forthefoption,ifthespecified
filenamehasnodeviceordirectorypathinformationinit,gawklooksinthecurrentdirectoryfirst,theninthedirectory
specifiedbythetranslationofAWK_LIBRARYifthefileisnotfound.If,aftersearchinginbothdirectories,thefilestillis
notfound,gawkappendsthesuffix.awktothefilenameandretriesthefilesearch.IfAWK_LIBRARYhasnodefinition,a
defaultvalueofSYS$LIBRARY:isusedforit.
Next:VMSGNV,Previous:VMSInstallationDetails,Up:VMSInstallation[Contents][Index]
B.3.2.4RunninggawkonVMS
CommandlineparsingandquotingconventionsaresignificantlydifferentonVMS,soexamplesinthisWebpageor
fromothersourcesoftenneedminorchanges.Theyareminorthough,andallawkprogramsshouldruncorrectly.
Hereareacoupleoftrivialtests:
$gawk"BEGIN{print""Hello,World!""}"
$gawk"W"version
!couldalsobe"Wversion"or"Wversion"
Notethatuppercaseandmixedcasetextmustbequoted.
TheVMSportofgawkincludesaDCLstyleinterfaceinadditiontotheoriginalshellstyleinterface(seethehelpentry
fordetails).Onesideeffectofdualcommandlineparsingisthatifthereisonlyasingleparameter(asinthequoted
stringprogram),thecommandbecomesambiguous.Toworkaroundthis,thenormallyoptionalflagisrequiredto
forceUnixstyleparsingratherthanDCLparsing.Ifanyotherdashtypeoptions(ormultipleparameterssuchasdatafiles
toprocess)arepresent,thereisnoambiguityandcanbeomitted.
TheexitvalueisaUnixstylevalueandisencodedintoaVMSexitstatusvaluewhentheprogramexits.
TheVMSseveritybitswillbesetbasedontheexitvalue.Afailureisindicatedby1,andVMSsetstheERRORstatus.A
fatalerrorisindicatedby2,andVMSsetstheFATALstatus.AllothervalueswillhavetheSUCCESSstatus.Theexitvalue
isencodedtocomplywithVMScodingstandardsandwillhavetheC_FACILITY_NOof0x350000withtheconstant0xA000
addedtothenumbershiftedoverby3bitstomakeroomfortheseveritycodes.
ToextracttheactualgawkexitcodefromtheVMSstatus,use:
unix_status=(vms_status.and.&x7f8)/8
ACprogramthatusesexec()tocallgawkwillgettheoriginalUnixstyleexitvalue.
OlderversionsofgawkforVMStreatedaUnixexitcode0as1,afailureas2,afatalerroras4,andpassedalltheother
numbersthrough.ThisviolatedtheVMSexitstatuscodingrequirements.
VAX/VMSfloatingpointusesunbiasedrounding.SeeRoundFunction.
VMSreportstimevaluesinGMTunlessoneoftheSYS$TIMEZONE_RULEorTZlogicalnamesisset.Olderversionsof
VMS,suchasVAX/VMS7.3,donotsettheselogicalnames.
Thedefaultsearchpath,whenlookingforawkprogramfilesspecifiedbythefoption,is"SYS$DISK:[],AWK_LIBRARY:".
ThelogicalnameAWKPATHcanbeusedtooverridethisdefault.TheformatofAWKPATHisacommaseparatedlistof
directoryspecifications.Whendefiningit,thevalueshouldbequotedsothatitretainsasingletranslationandnota
multitranslationRMSsearchlist.
Next:VMSOldGawk,Previous:VMSRunning,Up:VMSInstallation[Contents][Index]
371/479
29/7/2016
TheGNUAwkUsersGuide
B.3.2.5TheVMSGNVProject
TheVMSGNVpackageprovidesabuildenvironmentsimilartoPOSIXwithportsofacollectionofopensourcetools.
ThegawkfoundintheGNVbasekitisanolderport.Currently,theGNVprojectisbeingreorganizedtosupply
individualPCSIpackagesforeachcomponent.Seehttps://sourceforge.net/p/gnv/wiki/InstallingGNVPackages/.
ThenormalbuildprocedureforgawkproducesaprogramthatissuitableforusewithGNV.
Thefilevms/gawk_build_steps.txtinthedistributiondocumentstheprocedureforbuildingaVMSPCSIkitthatis
compatiblewithGNV.
Previous:VMSGNV,Up:VMSInstallation[Contents][Index]
B.3.2.6SomeVMSSystemsHaveAnOldVersionofgawk
SomeversionsofVMShaveanoldversionofgawk.Toaccessit,defineasymbol,asfollows:
$gawk:==$sys$common:[syshlp.examples.tcpip.snmp]gawk.exe
Thisisapparentlyversion2.15.6,whichisextremelyold.Werecommendcompilingandusingthecurrentversion.
Next:OtherVersions,Previous:NonUnixInstallation,Up:Installation[Contents][Index]
B.4ReportingProblemsandBugs
Thereisnothingmoredangerousthanaboredarchaeologist.
DouglasAdams,TheHitchhikersGuidetotheGalaxy
Ifyouhaveproblemswithgawkorthinkthatyouhavefoundabug,reportittothedeveloperswecannotpromisetodo
anything,butwemightwellwanttofixit.
Beforereportingabug,makesureyouhavereallyfoundagenuinebug.Carefullyrereadthedocumentationandseeifit
saysyoucandowhatyouretryingtodo.Ifitsnotclearwhetheryoushouldbeabletodosomethingornot,reportthat
tooitsabuginthedocumentation!
Beforereportingabugortryingtofixityourself,trytoisolateittothesmallestpossibleawkprogramandinputdatafile
thatreproducetheproblem.Thensendustheprogramanddatafile,someideaofwhatkindofUnixsystemyoure
using,thecompileryouusedtocompilegawk,andtheexactresultsgawkgaveyou.Alsosaywhatyouexpectedtooccur
thishelpsusdecidewhethertheproblemisreallyinthedocumentation.
Makesuretoincludetheversionnumberofgawkyouareusing.Youcangetthisinformationwiththecommandgawk
version.
Onceyouhaveapreciseproblemdescription,sendemailtobuggawkatgnudotorg.
Thegawkmaintainerssubscribetothisaddress,andthustheywillreceiveyourbugreport.Althoughyoucansendmail
tothemaintainersdirectly,thebugreportingaddressispreferredbecausetheemaillistisarchivedattheGNUProject.
AllemailmustbeinEnglish.Thisistheonlylanguageunderstoodincommonbyallthemaintainers.
CAUTION:DonottrytoreportbugsingawkbypostingtotheUsenet/Internetnewsgroupcomp.lang.awk.
Thegawkdevelopersdooccasionallyreadthisnewsgroup,butthereisnoguaranteethatwewillseeyour
posting.Thestepsdescribedherearetheonlyofficiallyrecognizedwayforreportingbugs.Really.
NOTE:ManydistributionsofGNU/LinuxandthevariousBSDbasedoperatingsystemshavetheirown
bugreportingsystems.Ifyoureportabugusingyourdistributionsbugreportingsystem,youshouldalso
372/479
29/7/2016
TheGNUAwkUsersGuide
sendacopytobuggawkatgnudotorg.
Thisisfortworeasons.First,althoughsomedistributionsforwardbugreportsupstreamtotheGNU
mailinglist,manydont,sothereisagoodchancethatthegawkmaintainerswontevenseethebugreport!
Second,mailtotheGNUlistisarchived,andhavingeverythingattheGNUProjectkeepsthingsself
containedandnotdependentonotherorganizations.
Nonbugsuggestionsarealwayswelcomeaswell.Ifyouhavequestionsaboutthingsthatareunclearinthe
documentationorarejustobscurefeatures,askonthebuglistwewilltrytohelpyououtifwecan.
IfyoufindbugsinoneofthenonUnixportsofgawk,sendanemailtothebuglist,withacopytothepersonwho
maintainsthatport.Themaintainersarenamedinthefollowinglist,aswellasintheREADMEfileinthegawkdistribution.
InformationintheREADMEfileshouldbeconsideredauthoritativeifitconflictswiththisWebpage.
Thepeoplemaintainingthevariousgawkportsare:
UnixandPOSIXsystems ArnoldRobbins,arnoldatskeevedotcom
MSDOSwithDJGPP
ScottDeifik,scottddotmailatsbcglobaldotnet
MSWindowswithMinGW EliZaretskii,elizatgnudotorg
OS/2
AndreasBuening,andreasdotbueningatnexgodotde
VMS
JohnMalmberg,wb8tywatqsl.net
z/OS(OS/390)
DavePitts,dpittsatcozxdotcom
IfyourbugisalsoreproducibleunderUnix,sendacopyofyourreporttothebuggawkatgnudotorgemaillistas
well.
Next:Installationsummary,Previous:Bugs,Up:Installation[Contents][Index]
B.5OtherFreelyAvailableawkImplementations
Itskindoffuntoputcommentslikethisinyourawkcode:
//DoC++commentswork?answer:yes!ofcourse
MichaelBrennan
Thereareanumberofotherfreelyavailableawkimplementations.Thissectionbrieflydescribeswheretogetthem:
Unixawk
BrianKernighan,oneoftheoriginaldesignersofUnixawk,hasmadehisimplementationofawkfreelyavailable.
Youcanretrievethisversionviahishomepage.Itisavailableinseveralarchiveformats:
Shellarchive
http://www.cs.princeton.edu/~bwk/btl.mirror/awk.shar
Compressedtarfile
http://www.cs.princeton.edu/~bwk/btl.mirror/awk.tar.gz
Zipfile
http://www.cs.princeton.edu/~bwk/btl.mirror/awk.zip
YoucanalsoretrieveitfromGitHub:
gitclonegit://github.com/onetrueawk/awkbwkawk
373/479
29/7/2016
TheGNUAwkUsersGuide
ThiscommandcreatesacopyoftheGitrepositoryinadirectorynamedbwkawk.Ifyouleavethatargumentoffthe
gitcommandline,therepositorycopyiscreatedinadirectorynamedawk.
ThisversionrequiresanISOC(1990standard)compilertheCcompilerfromGCC(theGNUCompiler
Collection)worksquitenicely.
SeeCommonExtensions,foralistofextensionsinthisawkthatarenotinPOSIXawk.
Asasidenote,DanBornsteinhascreatedaGitrepositorytrackingalltheversionsofBWKawkthathecould
find.Itsavailableatgit://github.com/danfuzz/onetrueawk.
mawk
MichaelBrennanwroteanindependentimplementationofawk,calledmawk.ItisavailableundertheGPL(see
Copying),justasgawkis.
Theoriginaldistributionsiteforthemawksourcecodenolongerhasit.Acopyisavailableat
http://www.skeeve.com/gawk/mawk1.3.3.tar.gz.
In2009,ThomasDickeytookonmawkmaintenance.Basicinformationisavailableontheprojectswebpage.The
downloadURLishttp://invisibleisland.net/datafiles/release/mawk.tar.gz.
Onceyouhaveit,gunzipmaybeusedtodecompressthisfile.Installationissimilartogawks(seeUnix
Installation).
SeeCommonExtensions,foralistofextensionsinmawkthatarenotinPOSIXawk.
awka
WrittenbyAndrewSumner,awkatranslatesawkprogramsintoC,compilesthem,andlinksthemwithalibraryof
functionsthatprovidethecoreawkfunctionality.Italsohasanumberofextensions.
TheawktranslatorisreleasedundertheGPL,andthelibraryisundertheLGPL.
Togetawka,gotohttp://sourceforge.net/projects/awka.
Theprojectseemstobefrozennonewcodechangeshavebeenmadesinceapproximately2001.
pawk
NelsonH.F.BeebeattheUniversityofUtahhasmodifiedBWKawktoprovidetimingandprofilinginformation.
Itisdifferentfromgawkwiththeprofileoption(seeProfiling)inthatitusesCPUbasedprofiling,notline
countprofiling.Youmayfinditateitherftp://ftp.math.utah.edu/pub/pawk/pawk20030606.tar.gzor
http://www.math.utah.edu/pub/pawk/pawk20030606.tar.gz.
BusyBoxawk
BusyBoxisaGPLlicensedprogramprovidingsmallversionsofmanyapplicationswithinasingleexecutable.It
isaimedatembeddedsystems.ItincludesafullimplementationofPOSIXawk.Whenbuildingit,becarefulnot
todomakeinstallasitwilloverwritecopiesofotherapplicationsinyour/usr/local/bin.Formore
information,seetheprojectshomepage.
TheOpenSolarisPOSIXawk
Theversionsofawkin/usr/xpg4/binand/usr/xpg6/binonSolarisaremoreorlessPOSIXcompliant.Theyare
basedontheawkfromMorticeKernSystemsforPCs.Wewereabletomakethiscodecompileandworkunder
GNU/Linuxwith12hoursofwork.Makingitmoregenerallyportable(usingGNUAutoconfand/orAutomake)
wouldtakemorework,andthishasnotbeendone,atleasttoourknowledge.
ThesourcecodeusedtobeavailablefromtheOpenSolariswebsite.However,thatprojectwasendedandthe
374/479
29/7/2016
TheGNUAwkUsersGuide
websiteshutdown.Fortunately,theIllumosprojectmakesthisimplementationavailable.Youcanviewthefiles
oneatatimefromhttps://github.com/joyent/illumosjoyent/blob/master/usr/src/cmd/awk_xpg4.
jawk
ThisisaninterpreterforawkwritteninJava.Itclaimstobeafullinterpreter,althoughbecauseitusesJava
facilitiesforI/Oandforregexpmatching,thelanguageitsupportsisdifferentfromPOSIXawk.Moreinformation
isavailableontheprojectshomepage.
Libmawk
Thisisanembeddableawkinterpreterderivedfrommawk.Formoreinformation,see
http://repo.hu/projects/libmawk/.
pawk
ThisisaPythonmodulethatclaimstobringawklikefeaturestoPython.Seehttps://github.com/alecthomas/pawk
formoreinformation.(ThisisnotrelatedtoNelsonBeebesmodifiedversionofBWKawk,describedearlier.)
QSEawk
Thisisanembeddableawkinterpreter.Formoreinformation,seehttp://code.google.com/p/qse/and
http://awk.info/?tools/qse.
QTawk
ThisisanindependentimplementationofawkdistributedundertheGPL.Ithasalargenumberofextensionsover
standardawkandmaynotbe100%syntacticallycompatiblewithit.Seehttp://www.quiktrim.org/QTawk.htmlfor
moreinformation,includingthemanualandadownloadlink.
Theprojectmayalsobefrozennonewcodechangeshavebeenmadesinceapproximately2008.
Otherversions
SeealsotheVersionsandimplementationssectionoftheWikipediaarticleonawkforinformationonadditional
versions.
Previous:OtherVersions,Up:Installation[Contents][Index]
B.6Summary
ThegawkdistributionisavailablefromtheGNUProjectsmaindistributionsite,ftp.gnu.org.Thecanonicalbuild
recipeis:
wgethttp://ftp.gnu.org/gnu/gawk/gawk4.1.2.tar.gz
cdgawk4.1.2
./configure&&make&&makecheck
gawkmaybebuiltonnonPOSIXsystemsaswell.ThecurrentlysupportedsystemsareMSWindowsusing
DJGPP,MSYS,MinGW,andCygwin,OS/2usingEMX,andbothVax/VMSandOpenVMS.Instructionsfor
eachsystemareincludedinthisappendix.
Bugreportsshouldbesentviaemailtobuggawk@gnu.org.BugreportsshouldbeinEnglishandshouldinclude
theversionofgawk,howitwascompiled,andashortprogramanddatafilethatdemonstratetheproblem.
Thereareanumberofotherfreelyavailableawkimplementations.ManyarePOSIXcompliantothersarelessso.
Next:BasicConcepts,Previous:Installation,Up:Top[Contents][Index]
375/479
29/7/2016
TheGNUAwkUsersGuide
Thisappendixcontainsinformationmainlyofinteresttoimplementersandmaintainersofgawk.Everythinginitapplies
specificallytogawkandnottootherimplementations.
CompatibilityMode:
Howtodisablecertaingawkextensions.
Additions:
MakingAdditionsTogawk.
FutureExtensions:
Newfeaturesthatmaybeimplementedoneday.
ImplementationLimitations: Somelimitationsoftheimplementation.
ExtensionDesign:
DesignnotesabouttheextensionAPI.
OldExtensionMechanism: Somecompatibilityforoldextensions.
Notessummary:
Summaryofimplementationnotes.
Next:Additions,Up:Notes[Contents][Index]
C.1DownwardCompatibilityandDebugging
SeePOSIX/GNU,forasummaryoftheGNUextensionstotheawklanguageandprogram.Allofthesefeaturescanbe
turnedoffbyinvokinggawkwiththetraditionaloptionorwiththeposixoption.
IfgawkiscompiledfordebuggingwithDDEBUG,thenthereisonemoreoptionavailableonthecommandline:
Y
parsedebug
Printouttheparsestackinformationastheprogramisbeingparsed.
Thisoptionisintendedonlyforseriousgawkdevelopersandnotforthecasualuser.Itprobablyhasnotevenbeen
compiledintoyourversionofgawk,sinceitslowsdownexecution.
Next:FutureExtensions,Previous:CompatibilityMode,Up:Notes[Contents][Index]
C.2MakingAdditionstogawk
Ifyoufindthatyouwanttoenhancegawkinasignificantfashion,youareperfectlyfreetodoso.Thatisthepointof
havingfreesoftwarethesourcecodeisavailableandyouarefreetochangeitasyouwant(seeCopying).
Thissectiondiscussesthewaysyoumightwanttochangegawkaswellasanyconsiderationsyoushouldbearinmind.
AccessingTheSource: AccessingtheGitrepository.
AddingCode:
Addingcodetothemainbodyofgawk.
NewPorts:
Portinggawktoanewoperatingsystem.
DerivedFiles:
WhyderivedfilesarekeptintheGitrepository.
Next:AddingCode,Up:Additions[Contents][Index]
C.2.1AccessingThegawkGitRepository
AsgawkisFreeSoftware,thesourcecodeisalwaysavailable.GawkDistribution,describeshowtogetandbuildthe
formal,releasedversionsofgawk.
However,ifyouwanttomodifygawkandcontributebackyourchanges,youwillprobablywishtoworkwiththe
developmentversion.Todoso,youwillneedtoaccessthegawksourcecoderepository.Thecodeismaintainedusing
theGitdistributedversioncontrolsystem.Youwillneedtoinstallitifyoursystemdoesnthaveit.Onceyouhavedone
376/479
29/7/2016
TheGNUAwkUsersGuide
so,usethecommand:
gitclonegit://git.savannah.gnu.org/gawk.git
Thisclonesthegawkrepository.IfyouarebehindafirewallthatdoesnotallowyoutousetheGitnativeprotocol,you
canstillaccesstherepositoryusing:
gitclonehttp://git.savannah.gnu.org/r/gawk.git
Onceyouhavemadechanges,youcanusegitdifftoproduceapatch,andsendthattothegawkmaintainersee
Bugs,forhowtodothat.
OnceuponatimetherewasGitCVSgatewayforusebypeoplewhocouldnotinstallGit.However,thisgatewayno
longerworks,soyoumayhavebetterluckusingamoremodernversioncontrolsystemlikeBazaar,thathasaGitplug
inforworkingwithGitrepositories.
Next:NewPorts,Previous:AccessingTheSource,Up:Additions[Contents][Index]
C.2.2AddingNewFeatures
Youarefreetoaddanynewfeaturesyouliketogawk.However,ifyouwantyourchangestobeincorporatedintothe
gawkdistribution,thereareseveralstepsthatyouneedtotakeinordertomakeitpossibletoincludethem:
1.Beforebuildingthenewfeatureintogawkitself,considerwritingitasanextension(seeDynamicExtensions).If
thatsnotpossible,continuewiththerestofthestepsinthislist.
2.Bepreparedtosigntheappropriatepaperwork.InorderfortheFSFtodistributeyourchanges,youmusteither
placethosechangesinthepublicdomainandsubmitasignedstatementtothateffect,orassignthecopyrightin
yourchangestotheFSF.Bothoftheseactionsareeasytodoandmanypeoplehavedonesoalready.Ifyouhave
questions,pleasecontactme(seeBugs),orassignatgnudotorg.
3.Getthelatestversion.Itismucheasierformetointegratechangesiftheyarerelativetothemostrecent
distributedversionofgawk,orbetteryet,relativetothelatestcodeintheGitrepository.Ifyourversionofgawkis
veryold,Imaynotbeabletointegrateyourchangesatall.(SeeGetting,forinformationongettingthelatest
versionofgawk.)
4.FollowtheGNUCodingStandards.ThisdocumentdescribeshowGNUsoftwareshouldbewritten.Ifyou
haventreadit,pleasedoso,preferablybeforestartingtomodifygawk.(TheGNUCodingStandardsareavailable
fromtheGNUProjectswebsite.Texinfo,Info,andDVIversionsarealsoavailable.)
5.Usethegawkcodingstyle.TheCcodeforgawkfollowstheinstructionsintheGNUCodingStandards,with
minorexceptions.ThecodeisformattedusingthetraditionalK&Rstyle,particularlyasregardstothe
placementofbracesandtheuseofTABs.Inbrief,thecodingrulesforgawkareasfollows:
UseANSI/ISOstyle(prototype)functionheaderswhendefiningfunctions.
Putthenameofthefunctionatthebeginningofitsownline.
Putthereturntypeofthefunction,evenifitisint,onthelineabovethelinewiththenameandarguments
ofthefunction.
Putspacesaroundparenthesesusedincontrolstructures(if,while,for,do,switch,andreturn).
Donotputspacesinfrontofparenthesesusedinfunctioncalls.
PutspacesaroundallCoperatorsandaftercommasinfunctioncalls.
Donotusethecommaoperatortoproducemultiplesideeffects,exceptinforloopinitializationand
incrementparts,andinmacrobodies.
UserealTABsforindenting,notspaces.
UsetheK&Rbracelayoutstyle.
UsecomparisonsagainstNULLand'\0'intheconditionsofif,while,andforstatements,aswellasinthe
casesofswitchstatements,insteadofjusttheplainpointerorcharactervalue.
Usetrueandfalseforboolvalues,theNULLsymbolicconstantforpointervalues,andthecharacter
constant'\0'whereappropriate,insteadof1and0.
Provideonelinedescriptivecommentsforeachfunction.
Donotusethealloca()functionforallocatingmemoryoffthestack.Itsusecausesmoreportability
troublethanisworththeminorbenefitofnothavingtofreethestorage.Instead,usemalloc()andfree().
377/479
29/7/2016
TheGNUAwkUsersGuide
Donotusecomparisonsoftheform!strcmp(a,b)orsimilar.AsHenrySpenceroncesaid,strcmp()is
notaboolean!Instead,usestrcmp(a,b)==0.
Ifaddingnewbitflagvalues,useexplicithexadecimalconstants(0x001,0x002,0x004,andsonon)instead
ofshiftingoneleftbysuccessiveamounts((1<<0),(1<<1),andsoon).
NOTE:IfIhavetoreformatyourcodetofollowthecodingstyleusedingawk,Imaynotbotherto
integrateyourchangesatall.
6.Updatethedocumentation.Alongwithyournewcode,pleasesupplynewsectionsand/orchaptersforthisWeb
page.Ifatallpossible,pleaseuserealTexinfo,insteadofjustsupplyingunformattedASCIItext(althougheven
thatisbetterthannodocumentationatall).ConventionstobefollowedinGAWK:EffectiveAWKProgramming
areprovidedafterthe@byeattheendoftheTexinfosourcefile.Ifpossible,pleaseupdatethemanpageaswell.
Youwillalsohavetosignpaperworkforyourdocumentationchanges.
7.Submitchangesasunifieddiffs.UsediffurNtocomparetheoriginalgawksourcetreewithyourversion.
IrecommendusingtheGNUversionofdiff,orbestofall,gitdifforgitformatpatch.Sendtheoutput
producedbydifftomewhenyousubmityourchanges.(SeeBugs,fortheelectronicmailinformation.)
Usingthisformatmakesiteasyformetoapplyyourchangestothemasterversionofthegawksourcecode(using
patch).IfIhavetoapplythechangesmanually,usingatexteditor,Imaynotdoso,particularlyiftherearelotsof
changes.
8.IncludeanentryfortheChangeLogfilewithyoursubmission.ThishelpsfurtherminimizetheamountofworkI
havetodo,makingiteasierformetoacceptpatches.Itissimplestifyoujustmakethispartofyourdiff.
Althoughthissoundslikealotofwork,pleaserememberthatwhileyoumaywritethenewcode,Ihavetomaintainit
andsupportit.Ifitisntpossibleformetodothatwithaminimumofextrawork,thenIprobablywillnot.
Next:DerivedFiles,Previous:AddingCode,Up:Additions[Contents][Index]
C.2.3PortinggawktoaNewOperatingSystem
Ifyouwanttoportgawktoanewoperatingsystem,thereareseveralsteps:
1.Followtheguidelinesintheprevioussectionconcerningcodingstyle,submissionofdiffs,andsoon.
2.Bepreparedtosigntheappropriatepaperwork.InorderfortheFSFtodistributeyourcode,youmusteitherplace
yourcodeinthepublicdomainandsubmitasignedstatementtothateffect,orassignthecopyrightinyourcode
totheFSF.Bothoftheseactionsareeasytodoandmanypeoplehavedonesoalready.Ifyouhavequestions,
pleasecontactme,orgnu@gnu.org.
3.Whendoingaport,bearinmindthatyourcodemustcoexistpeacefullywiththerestofgawkandtheotherports.
Avoidgratuitouschangestothesystemindependentpartsofthecode.Ifatallpossible,avoidsprinkling#ifdefs
justforyourportthroughoutthecode.
Ifthechangesneededforaparticularsystemaffecttoomuchofthecode,Iprobablywillnotacceptthem.Insuch
acase,youcan,ofcourse,distributeyourchangesonyourown,aslongasyoucomplywiththeGPL(see
Copying).
4.Anumberofthefilesthatcomewithgawkaremaintainedbyotherpeople.Thus,youshouldnotchangethem
unlessitisforaverygoodreasoni.e.,changesarenotoutofthequestion,butchangestothesefilesare
scrutinizedextracarefully.Thefilesaredfa.c,dfa.h,getopt.c,getopt.h,getopt1.c,getopt_int.h,gettext.h,
regcomp.c,regex.c,regex.h,regex_internal.c,regex_internal.h,andregexec.c.
5.AnumberofotherfilesareprovidedbytheGNUAutotools(Autoconf,Automake,andGNUgettext).You
shouldnotchangethemeither,unlessitisforaverygoodreason.ThefilesareABOUTNLS,config.guess,
config.rpath,config.sub,depcomp,INSTALL,installsh,missing,mkinstalldirs,xalloc.h,andylwrap.
6.Bewillingtocontinuetomaintaintheport.NonUnixoperatingsystemsaresupportedbyvolunteerswho
maintainthecodeneededtocompileandrungawkontheirsystems.Ifnoonevolunteerstomaintainaport,it
378/479
29/7/2016
TheGNUAwkUsersGuide
becomesunsupportedanditmaybenecessarytoremoveitfromthedistribution.
7.Supplyanappropriategawkmisc.???file.Eachporthasitsowngawkmisc.???thatimplementscertainoperating
systemspecificfunctions.Thisiscleanerthanaplethoraof#ifdefsscatteredthroughoutthecode.The
gawkmisc.cinthemainsourcedirectoryincludestheappropriategawkmisc.???filefromeachsubdirectory.Be
suretoupdateitaswell.
Eachportsgawkmisc.???filehasasuffixreminiscentofthemachineoroperatingsystemfortheportfor
example,pc/gawkmisc.pcandvms/gawkmisc.vms.Theuseofseparatesuffixes,insteadofplaingawkmisc.c,makes
itpossibletomovefilesfromaportssubdirectoryintothemainsubdirectory,withoutaccidentallydestroyingthe
realgawkmisc.cfile.(Currently,thisisonlyanissueforthePCoperatingsystemports.)
8.SupplyaMakefileaswellasanyotherCsourceandheaderfilesthatarenecessaryforyouroperatingsystem.All
yourcodeshouldbeinaseparatesubdirectory,withanamethatisthesameas,orreminiscentof,eitheryour
operatingsystemorthecomputersystem.Ifpossible,trytostructurethingssothatitisnotnecessarytomove
filesoutofthesubdirectoryintothemainsourcedirectory.Ifthatisnotpossible,thenbesuretoavoidusing
namesforyourfilesthatduplicatethenamesoffilesinthemainsourcedirectory.
9.Updatethedocumentation.Pleasewriteasection(orsections)forthisWebpagedescribingtheinstallationand
compilationstepsneededtocompileand/orinstallgawkforyoursystem.
Followingthesestepsmakesitmucheasiertointegrateyourchangesintogawkandhavethemcoexisthappilywithother
operatingsystemscodethatisalreadythere.
Inthecodethatyousupplyandmaintain,feelfreetouseacodingstyleandbracelayoutthatsuitsyourtaste.
Previous:NewPorts,Up:Additions[Contents][Index]
C.2.4WhyGeneratedFilesAreKeptInGit
IfyoulookatthegawksourceintheGitrepository,youwillnoticethatitincludesfilesthatareautomaticallygenerated
byGNUinfrastructuretools,suchasMakefile.infromAutomakeandevenconfigurefromAutoconf.
ThisisdifferentfrommanyFreeSoftwareprojectsthatdonotstorethederivedfiles,becausethatkeepstherepository
lesscluttered,anditiseasiertoseethesubstantivechangeswhencomparingversionsandtryingtounderstandwhat
changedbetweencommits.
However,thereareseveralreasonswhythegawkmaintainerlikestohaveeverythingintherepository.
First,becauseitistheneasytoreproduceanygivenversioncompletely,withoutrelyingupontheavailabilityof(older,
likelyobsolete,andmaybeevenimpossibletofind)othertools.
Asanextremeexample,ifyouevereventhinkabouttryingtocompile,oh,say,theV7awk,youwilldiscoverthatnot
onlydoyouhavetobootstraptheV7yacctodoso,butyoualsoneedtheV7lex.Andthelatterisprettymuch
impossibletobringuponamodernGNU/Linuxsystem.112
(Or,letssaygawk1.2requiredbisonwhateveritwasin1989andthattherewasnoawkgram.cfileintherepository.Is
thereaguaranteethatwecouldfindthatbisonversion?Orthatitwouldbuild?)
Iftherepositoryhasallthegeneratedfiles,thenitseasytojustcheckthemoutandbuild.(Oreasier,dependingupon
howfarbackwego.)
Andthatbringsustothesecond(andstronger)reasonwhyallthefilesreallyneedtobeinGit.Itboilsdowntowhodo
youcatertothegawkdeveloper(s),ortheuserwhojustwantstocheckoutaversionandtryitout?
Thegawkmaintainerwantsittobepossibleforanyinterestedawkuserintheworldtojustclonetherepository,check
outthebranchofinterestandbuildit.Withouttheirhavingtohavethecorrectversion(s)oftheautotools.113Thatisthe
pointofthebootstrap.shfile.Ittouchesthevariousotherfilesintherightordersuchthat
379/479
29/7/2016
TheGNUAwkUsersGuide
#ThecanonicalincantationforbuildingGNUsoftware:
./bootstrap.sh&&./configure&&make
willjustwork.
ThisisextremelyimportantforthemasterandgawkX.Ystablebranches.
Further,thegawkmaintainerwouldarguethatitsalsoimportantforthegawkdevelopers.Whenhetriedtocheckoutthe
xgawkbranch114tobuildit,hecouldnt.(Noltmain.shfile,andhehadnoideahowtocreateit,andthatwasnotthe
onlyproblem.)
Hefeltextremelyfrustrated.Withrespecttothatbranch,themaintainerisnodifferentthanJaneUserwhowantstotry
tobuildgawk4.1stableormasterfromtherepository.
Thus,themaintainerthinksthatitsnotjustimportant,butcritical,thatforanygivenbranch,theaboveincantationjust
works.
Athirdreasontohaveallthefilesisthatwithoutthem,usinggitbisecttotrytofindthecommitthatintroduceda
bugisexceedinglydifficult.Themaintainertriedtodothatonanotherprojectthatrequiresrunningbootstrapping
scriptsjusttocreateconfigureandsoonitwasreallypainful.Whentherepositoryisselfcontained,usinggitbisect
initisveryeasy.
Whataresomeoftheconsequencesand/oractionstotake?
1.Wedontmindthattherearedifferingfilesinthedifferentbranchesasaresultofdifferentversionsofthe
autotools.
1.Itsthemaintainersjobtomergethemandhewilldealwithit.
2.Heisreallygoodatgitdiffxy>/tmp/diff1;gvim/tmp/diff1toremovethediffsthatarentof
interestinordertoreviewcode.
2.ItwouldcertainlyhelpifeveryoneusedthesameversionsoftheGNUtoolsashedoes,whichingeneralarethe
latestreleasedversionsofAutomake,Autoconf,bison,andGNUgettext.
Installingfromsourceisquiteeasy.Itshowthemaintainerworkedforyears(andstillworks).Hehad
/usr/local/binatthefrontofhisPATHandjustdid:
wgethttp://ftp.gnu.org/gnu/package/packagex.y.z.tar.gz
tarxpzvfpackagex.y.z.tar.gz
cdpackagex.y.z
./configure&&make&&makecheck
makeinstall#asroot
Mostoftheabovewasoriginallywrittenbythemaintainertoothergawkdevelopers.Itraisedtheobjectionfromoneof
thedevelopersthatanybodypullingdownthesourcefromGitisnotanenduser.
However,thisisnottrue.Therearepowerawkuserswhocanbuildgawk(usingthemagicincantationshown
previously)butwhocantprograminC.Thus,themajorbranchesshouldbekeptbuildableallthetime.
Itwasthensuggestedthattherebeacronjobtocreatenightlytarballsofthesource.Here,theproblemisthatthere
aresourcetrees,correspondingtothevariousbranches!So,nightlytarballsarenttheanswer,especiallyasthe
repositorycangoforweekswithoutsignificantchangebeingintroduced.
Fortunately,theGitservercanmeetthisneed.Foranygivenbranchnamedbranchname,use:
wgethttp://git.savannah.gnu.org/cgit/gawk.git/snapshot/gawkbranchname.tar.gz
toretrieveasnapshotofthegivenbranch.
Next:ImplementationLimitations,Previous:Additions,Up:Notes[Contents][Index]
380/479
29/7/2016
TheGNUAwkUsersGuide
C.3ProbableFutureExtensions
AWKisalanguagesimilartoPERL,onlyconsiderablymoreelegant.
ArnoldRobbins
Hey!
LarryWall
TheTODOfileinthemasterbranchofthegawkGitrepositorylistspossiblefutureenhancements.Someoftheserelateto
thesourcecode,andotherstopossiblenewfeatures.Pleaseseethatfileforthelist.SeeAdditions,ifyouareinterested
intacklinganyoftheprojectslistedthere.
Next:ExtensionDesign,Previous:FutureExtensions,Up:Notes[Contents][Index]
C.4SomeLimitationsoftheImplementation
ThisfollowingtabledescribeslimitsofgawkonaUnixlikesystem(althoughitisvariableeventhen).Othersystems
mayhavedifferentlimits.
Item
Charactersinacharacterclass
Lengthofinputrecord
Lengthofoutputrecord
Lengthofsourceline
Numberoffieldsinarecord
Numberoffileredirections
Numberofinputrecordsinonefile
Numberofinputrecordstotal
Numberofpiperedirections
Numericvalues
Sizeofafield
Sizeofaliteralstring
Sizeofaprintfstring
Limit
2^(numberofbitsperbyte)
MAX_INT
Unlimited
Unlimited
MAX_LONG
Unlimited
MAX_LONG
MAX_LONG
min(numberofprocessesperuser,numberofopen
files)
Doubleprecisionfloatingpoint(ifnotusingMPFR)
MAX_INT
MAX_INT
MAX_INT
Next:OldExtensionMechanism,Previous:ImplementationLimitations,Up:Notes[Contents][Index]
C.5ExtensionAPIDesign
ThissectiondocumentsthedesignoftheextensionAPI,includingadiscussionofsomeofthehistoryandproblemsthat
neededtobesolved.
Thefirstversionofextensionsforgawkwasdevelopedinthemid1990sandreleasedwithgawk3.1inthelate1990s.
Thebasicmechanismsanddesignremainedunchangedforcloseto15years,until2012.
Theoldextensionmechanismuseddatatypesandfunctionsfromgawkitself,withacleverhacktoinstallextension
functions.
gawkincludedsomesampleextensions,ofwhichafewwerereallyuseful.However,itwasclearfromtheoutsetthatthe
extensionmechanismwasboltedontothesideandwasnotreallywellthoughtout.
381/479
29/7/2016
TheGNUAwkUsersGuide
OldExtensionProblems:
Problemswiththeoldmechanism.
ExtensionNewMechanismGoals: Goalsforthenewmechanism.
ExtensionOtherDesignDecisions: Someotherdesigndecisions.
ExtensionFutureGrowth:
Someroomforfuturegrowth.
Next:ExtensionNewMechanismGoals,Up:ExtensionDesign[Contents][Index]
C.5.1ProblemsWithTheOldMechanism
Theoldextensionmechanismhadseveralproblems:
Itdependedheavilyupongawkinternals.AnytimetheNODEstructure115changed,anextensionwouldhavetobe
recompiled.Furthermore,toreallywriteextensionsrequiredunderstandingsomethingaboutgawksinternal
functions.TherewassomedocumentationinthisWebpage,butitwasquiteminimal.
BeingabletocallintogawkfromanextensionrequiredlinkerfacilitiesthatarecommononUnixderivedsystems
butthatdidnotworkonMSWindowssystemsuserswantingextensionsonMSWindowshadtostaticallylink
themintogawk,eventhoughMSWindowssupportsdynamicloadingofsharedobjects.
TheAPIwouldchangeoccasionallyasgawkchangednocompatibilitybetweenversionswaseverofferedor
plannedfor.
Despitethedrawbacks,thexgawkprojectdevelopersforkedgawkanddevelopedseveralsignificantextensions.They
alsoenhancedgawksfacilitiesrelatingtofileinclusionandsharedobjectaccess.
AnewAPIwasdesiredforalongtime,butonlyin2012didthegawkmaintainerandthexgawkdevelopersfinallystart
workingonittogether.Moreinformationaboutthexgawkprojectisprovidedingawkextlib.
Next:ExtensionOtherDesignDecisions,Previous:OldExtensionProblems,Up:ExtensionDesign[Contents][Index]
C.5.2GoalsForANewMechanism
SomegoalsforthenewAPIwere:
TheAPIshouldbeindependentofgawkinternals.Changesingawkinternalsshouldnotbevisibletothewriterof
anextensionfunction.
TheAPIshouldprovidebinarycompatibilityacrossgawkreleasesaslongastheAPIitselfdoesnotchange.
TheAPIshouldenableextensionswritteninCorC++tohaveroughlythesameappearancetoawklevelcode
asawkfunctionsdo.Thismeansthatextensionsshouldhave:
Theabilitytoaccessfunctionparameters.
Theabilitytoturnanundefinedparameterintoanarray(callbyreference).
Theabilitytocreate,accessandupdateglobalvariables.
Easyaccesstoalltheelementsofanarrayatonce(arrayflattening)inordertoloopoveralltheelement
inaneasyfashionforCcode.
Theabilitytocreatearrays(includinggawkstruearraysofarrays).
Someadditionalimportantgoalswere:
TheAPIshoulduseonlyfeaturesinISOC90,sothatextensionscanbewrittenusingthewidestrangeofCand
C++compilers.Theheadershouldincludetheappropriate#ifdef__cplusplusandextern"C"magicsothata
C++compilercouldbeused.(IfusingC++,theruntimesystemhastobesmartenoughtocallanyconstructors
anddestructors,asgawkisaCprogram.Asofthiswriting,thishasnotbeentested.)
TheAPImechanismshouldnotrequireaccesstogawkssymbols116bythecompiletimeordynamiclinker,in
ordertoenablecreationofextensionsthatalsoworkonMSWindows.
Duringdevelopment,itbecameclearthattherewereotherfeaturesthatshouldbeavailabletoextensions,whichwere
alsosubsequentlyprovided:
382/479
29/7/2016
TheGNUAwkUsersGuide
ExtensionsshouldhavetheabilitytohookintogawksI/Oredirectionmechanism.Inparticular,thexgawk
developersprovidedasocalledopenhooktotakeoverreadingrecords.Duringdevelopment,thiswas
generalizedtoallowextensionstohookintoinputprocessing,outputprocessing,andtwowayI/O.
Anextensionshouldbeabletoprovideacallbackfunctiontoperformcleanupactionswhengawkexits.
Anextensionshouldbeabletoprovideaversionstringsothatgawksversionoptioncanprovideinformation
aboutextensionsaswell.
Therequirementtoavoidaccesstogawkssymbolsis,atfirstglance,adifficultonetomeet.
Onedesign,apparentlyusedbyPerlandRubyandmaybeothers,wouldbetomakethemainlinegawkcodeintoa
library,withthegawkutilityasmallCmain()functionlinkedagainstthelibrary.
Thisseemedlikethetailwaggingthedog,complicatingbuildandinstallationandmakingasimplecopyofthegawk
executablefromonesystemtoanother(oroneplacetoanotheronthesamesystem!)intoachancyoperation.
PatRankinsuggestedthesolutionthatwasadopted.SeeExtensionMechanismOutline,forthedetails.
Next:ExtensionFutureGrowth,Previous:ExtensionNewMechanismGoals,Up:ExtensionDesign[Contents][Index]
C.5.3OtherDesignDecisions
Asanarbitrarydesigndecision,extensionscanreadthevaluesofpredefinedvariablesandarrays(suchasARGVandFS),
butcannotchangethem,withtheexceptionofPROCINFO.
Thereasonforthisistopreventanextensionfunctionfromaffectingtheflowofanawkprogramoutsideitscontrol.
Whilearealawkfunctioncandowhatitlikes,thatisatthediscretionoftheprogrammer.Anextensionfunctionshould
provideaserviceormakeaCAPIavailableforusewithinawk,andnotmesswithFSorARGCandARGV.
Inaddition,itbecomeseasytostartdownaslipperyslope.Howmuchaccesstogawkfacilitiesdoextensionsneed?Do
theyneedgetline?Whataboutcallinggsub()orcompilingregularexpressions?Whataboutcallingintoawkfunctions?
(Thatwouldbemessy.)
Inordertoavoidtheseissues,thegawkdeveloperschosetostartwiththesimplest,mostbasicfeaturesthatarestilltruly
useful.
AnotherdecisionisthatalthoughgawkprovidesnicethingslikeMPFR,andarraysindexedinternallybyintegers,these
featuresarenotbeingbroughtouttotheAPIinordertokeepthingssimpleandclosetotraditionalawksemantics.(In
fact,arraysindexedinternallybyintegersaresotransparentthattheyarentevendocumented!)
Additionally,allfunctionsintheAPIcheckthattheirpointerinputparametersarenotNULL.Iftheyare,theyreturnan
error.(ItisagoodideaforextensioncodetoverifythatpointersreceivedfromgawkarenotNULL.Suchathingshould
nothappen,butthegawkdevelopersareonlyhuman,andtheyhavebeenknowntooccasionallymakemistakes.)
Withtime,theAPIwillundoubtedlyevolvethegawkdevelopersexpectthistobedrivenbyuserneeds.Fornow,the
currentAPIseemstoprovideaminimalyetpowerfulsetoffeaturesforcreatingextensions.
Previous:ExtensionOtherDesignDecisions,Up:ExtensionDesign[Contents][Index]
C.5.4RoomForFutureGrowth
TheAPIcanlaterbeexpanded,intwoways:
gawkpassesanextensionidintotheextensionwhenitfirstloadstheextension.Theextensionthenpassesthis
idbacktogawkwitheachfunctioncall.Thismechanismallowsgawktoidentifytheextensioncallingintoit,
shoulditneedtoknow.
Similarly,theextensionpassesanamespaceintogawkwhenitregisterseachextensionfunction.This
383/479
29/7/2016
TheGNUAwkUsersGuide
accommodatesapossiblefuturemechanismforgroupingextensionfunctionsandpossiblyavoidingname
conflicts.
Ofcourse,asofthiswriting,nodecisionshavebeenmadewithrespecttoanyoftheabove.
Next:Notessummary,Previous:ExtensionDesign,Up:Notes[Contents][Index]
C.6CompatibilityForOldExtensions
DynamicExtensions,describesthesupportedAPIandmechanismsforwritingextensionsforgawk.ThisAPIwas
introducedinversion4.1.However,formanyyearsgawkprovidedanextensionmechanismthatrequiredknowledgeof
gawkinternalsandthatwasnotaswelldesigned.
Inordertoprovideatransitionperiod,gawkversion4.1continuestosupporttheoriginalextensionmechanism.Thiswill
betrueforthelifeofexactlyonemajorrelease.Thissupportwillbewithdrawn,andremovedfromthesourcecode,at
thenextmajorrelease.
Briefly,originalstyleextensionsshouldbecompiledbyincludingtheawk.hheaderfileintheextensionsourcecode.
Additionally,youmustdefinetheidentifierGAWKwhenbuilding(useDGAWKwithUnixstylecompilers).Otherwise,
thedefinitionsingawkapi.hwillcauseconflictswiththoseinawk.handyourextensionwillnotcompile.
Justasinpreviousversions,youloadanoldstyleextensionwiththeextension()builtinfunction(whichisnot
otherwisedocumented).Thisfunctioninturnfindsandloadsthesharedobjectfilecontainingtheextensionandcallsits
dl_load()Croutine.
Becauseoriginalstyleandnewstyleextensionsusedifferentinitializationroutines(dl_load()versusdlload()),they
maysafelybeinstalledinthesamedirectory(tobefoundbyAWKLIBPATH)withoutconflict.
Thegawkdevelopmentteamstronglyrecommendsthatyouconvertanyoldextensionsthatyoumayhavetousethenew
APIdescribedinDynamicExtensions.
Previous:OldExtensionMechanism,Up:Notes[Contents][Index]
C.7Summary
gawksextensionscanbedisabledwitheitherthetraditionaloptionorwiththeposixoption.The
parsedebugoptionisavailableifgawkiscompiledwithDDEBUG.
ThesourcecodeforgawkismaintainedinapubliclyaccessibleGitrepository.Anyonemaycheckitoutandview
thesource.
Contributionstogawkarewelcome.Followingthestepsoutlinedinthischapterwillmakeiteasiertointegrate
yourcontributionsintothecodebase.Thisappliesbothtonewfeaturecontributionsandtoportstoadditional
operatingsystems.
gawkhassomelimitsgenerallythosethatareimposedbythemachinearchitecture.
TheextensionAPIdesignwasintendedtosolveanumberofproblemswiththepreviousextensionmechanism,
enablefeaturesneededbythexgawkproject,andprovidebinarycompatibilitygoingforward.
Thepreviousextensionmechanismisstillsupportedinversion4.1ofgawk,butitwillberemovedinthenext
majorrelease.
Next:Glossary,Previous:Notes,Up:Top[Contents][Index]
ThisappendixattemptstodefinesomeofthebasicconceptsandtermsthatareusedthroughouttherestofthisWeb
page.AsthisWebpageisspecificallyaboutawk,andnotaboutcomputerprogrammingingeneral,thecoveragehereis
384/479
29/7/2016
TheGNUAwkUsersGuide
bynecessityfairlycursoryandsimplistic.(Ifyouneedmorebackground,therearemanyotherintroductorytextsthat
youshouldrefertoinstead.)
BasicHighLevel: Thehighlevelview.
BasicDataTyping: Averyquickintrotodatatypes.
Next:BasicDataTyping,Up:BasicConcepts[Contents][Index]
D.1WhataProgramDoes
Atthemostbasiclevel,thejobofaprogramistoprocesssomeinputdataandproduceresults.SeeFigureD.1.
FigureD.1:GeneralProgramFlow
Theprograminthefigurecanbeeitheracompiledprogram117(suchasls),oritmaybeinterpreted.Inthelatter
case,amachineexecutableprogramsuchasawkreadsyourprogram,andthenusestheinstructionsinyourprogramto
processthedata.
Whenyouwriteaprogram,itusuallyconsistsofthefollowing,verybasicsetofsteps,asshowninFigureD.2:
FigureD.2:BasicProgramSteps
Initialization
Thesearethethingsyoudobeforeactuallystartingtoprocessdata,suchascheckingarguments,initializingany
datayouneedtoworkwith,andsoon.ThisstepcorrespondstoawksBEGINrule(seeBEGIN/END).
Ifyouwerebakingacake,thismightconsistoflayingoutallthemixingbowlsandthebakingpan,andmaking
sureyouhavealltheingredientsthatyouneed.
Processing
Thisiswheretheactualworkisdone.Yourprogramreadsdata,onelogicalchunkatatime,andprocessesitas
appropriate.
Inmostprogramminglanguages,youhavetomanuallymanagethereadingofdata,checkingtoseeifthereis
moreeachtimeyoureadachunk.awkspatternactionparadigm(seeGettingStarted)handlesthemechanicsof
thisforyou.
Inbakingacake,theprocessingcorrespondstotheactuallabor:breakingeggs,mixingtheflour,water,andother
ingredients,andthenputtingthecakeintotheoven.
385/479
29/7/2016
TheGNUAwkUsersGuide
CleanUp
Onceyouveprocessedallthedata,youmayhavethingsyouneedtodobeforeexiting.Thisstepcorrespondsto
awksENDrule(seeBEGIN/END).
Afterthecakecomesoutoftheoven,youstillhavetowrapitinplasticwraptokeepanyonefromtastingit,as
wellaswashthemixingbowlsandutensils.
Analgorithmisadetailedsetofinstructionsnecessarytoaccomplishatask,orprocessdata.Itismuchthesameasa
recipeforbakingacake.Programsimplementalgorithms.Often,itisuptoyoutodesignthealgorithmandimplement
it,simultaneously.
Thelogicalchunkswetalkedaboutpreviouslyarecalledrecords,similartotherecordsacompanykeepson
employees,aschoolkeepsforstudents,oradoctorkeepsforpatients.Eachrecordhasmanycomponentparts,suchas
firstandlastnames,dateofbirth,address,andsoon.Thecomponentpartsarereferredtoasthefieldsoftherecord.
Theactofreadingdataistermedinput,andthatofgeneratingresults,nottoosurprisingly,istermedoutput.Theyare
oftenreferredtotogetherasinput/output,andevenmoreoften,asI/Oforshort.(Youwillalsoseeinputand
outputusedasverbs.)
awkmanagesthereadingofdataforyou,aswellasthebreakingitupintorecordsandfields.Yourprogramsjobisto
tellawkwhattodowiththedata.Youdothisbydescribingpatternsinthedatatolookfor,andactionstoexecutewhen
thosepatternsareseen.Thisdatadrivennatureofawkprogramsusuallymakesthembotheasiertowriteandeasierto
read.
Previous:BasicHighLevel,Up:BasicConcepts[Contents][Index]
D.2DataValuesinaComputer
Inaprogram,youkeeptrackofinformationandvaluesinthingscalledvariables.Avariableisjustanameforagiven
value,suchasfirst_name,last_name,address,andsoon.awkhasseveralpredefinedvariables,andithasspecialnames
torefertothecurrentinputrecordandthefieldsoftherecord.Youmayalsogroupmultipleassociatedvaluesunderone
name,asanarray.
Data,particularlyinawk,consistsofeithernumericvalues,suchas42or3.1415927,orstringvalues.Stringvaluesare
essentiallyanythingthatsnotanumber,suchasaname.Stringsaresometimesreferredtoascharacterdata,sincethey
storetheindividualcharactersthatcomprisethem.Individualvariables,aswellasnumericandstringvariables,are
referredtoasscalarvalues.Groupsofvalues,suchasarrays,arenotscalars.
ComputerArithmetic,providedabasicintroductiontonumerictypes(integerandfloatingpoint)andhowtheyareused
inacomputer.Pleasereviewthatinformation,includinganumberofcaveatsthatwerepresented.
Whileyouareprobablyusedtotheideaofanumberwithoutavalue(i.e.,zero),ittakesabitmoregettingusedtothe
ideaofzerolengthcharacterdata.Nevertheless,suchathingexists.Itiscalledthenullstring.Thenullstringis
characterdatathathasnovalue.Inotherwords,itisempty.Itiswritteninawkprogramslikethis:"".
Humansareusedtoworkingindecimali.e.,base10.Inbase10,numbersgofrom0to9,andthenrolloverintothe
nextcolumn.(Remembergradeschool?42=4x10+2.)
Thereareothernumberbasesthough.Computerscommonlyusebase2orbinary,base8oroctal,andbase16or
hexadecimal.Inbinary,eachcolumnrepresentstwotimesthevalueinthecolumntoitsright.Eachcolumnmaycontain
eithera0ora1.Thus,binary1010represents(1x8)+(0x4)+(1x2)+(0x1),ordecimal10.Octalandhexadecimal
arediscussedmoreinNondecimalnumbers.
Attheverylowestlevel,computersstorevaluesasgroupsofbinarydigits,orbits.Moderncomputersgroupbitsinto
groupsofeight,calledbytes.Advancedapplicationssometimeshavetomanipulatebitsdirectly,andgawkprovides
functionsfordoingso.
386/479
29/7/2016
TheGNUAwkUsersGuide
Programsarewritteninprogramminglanguages.Hundreds,ifnotthousands,ofprogramminglanguagesexist.Oneof
themostpopularistheCprogramminglanguage.TheClanguagehadaverystronginfluenceonthedesignoftheawk
language.
TherehavebeenseveralversionsofC.ThefirstisoftenreferredtoasK&RC,aftertheinitialsofBrianKernighan
andDennisRitchie,theauthorsofthefirstbookonC.(DennisRitchiecreatedthelanguage,andBrianKernighanwas
oneofthecreatorsofawk.)
Inthemid1980s,aneffortbegantoproduceaninternationalstandardforC.Thisworkculminatedin1989,withthe
productionoftheANSIstandardforC.ThisstandardbecameanISOstandardin1990.In1999,arevisedISOC
standardwasapprovedandreleased.Whereitmakessense,POSIXawkiscompatiblewith1999ISOC.
Next:Copying,Previous:BasicConcepts,Up:Top[Contents][Index]
Glossary
Action
Aseriesofawkstatementsattachedtoarule.Iftherulespatternmatchesaninputrecord,awkexecutestherules
action.Actionsarealwaysenclosedinbraces.(SeeActionOverview.)
Ada
AprogramminglanguageoriginallydefinedbytheU.S.DepartmentofDefenseforembeddedprogramming.It
wasdesignedtoenforcegoodSoftwareEngineeringpractices.
AmazingawkAssembler
HenrySpencerattheUniversityofTorontowrotearetargetableassemblercompletelyassedandawkscripts.Itis
thousandsoflineslong,includingmachinedescriptionsforseveraleightbitmicrocomputers.Itisagoodexample
ofaprogramthatwouldhavebeenbetterwritteninanotherlanguage.Youcangetitfromhttp://awk.info/?
awk100/aaa.
AmazinglyWorkableFormatter(awf)
HenrySpencerattheUniversityofTorontowroteaformatterthatacceptsalargesubsetofthenroffmsand
nroffmanformattingcommands,usingawkandsh.Itisavailablefromhttp://awk.info/?tools/awf.
Anchor
Theregexpmetacharactersând$,whichforcethematchtothebeginningorendofthestring,respectively.
ANSI
TheAmericanNationalStandardsInstitute.Thisorganizationproducesmanystandards,amongthemthe
standardsfortheCandC++programminglanguages.Thesestandardsoftenbecomeinternationalstandardsas
well.SeealsoISO.
Argument
Anargumentcanbetwodifferentthings.Itcanbeanoptionorafilenamepassedtoacommandwhileinvoking
itfromthecommandline,oritcanbesomethingpassedtoafunctioninsideaprogram,e.g.insideawk.
Inthelattercase,anargumentcanbepassedtoafunctionintwoways.Eitheritisgiventothecalledfunctionby
value,i.e.,acopyofthevalueofthevariableismadeavailabletothecalledfunction,buttheoriginalvariable
cannotbemodifiedbythefunctionitselforitisgivenbyreference,i.e.,apointertotheinterestedvariableis
passedtothefunction,whichcanthendirectlymodifyit.Inawkscalarsarepassedbyvalue,andarraysarepassed
byreference.SeePassByValue/Reference.
387/479
29/7/2016
TheGNUAwkUsersGuide
Array
Agroupingofmultiplevaluesunderthesamename.Mostlanguagesjustprovidesequentialarrays.awkprovides
associativearrays.
Assertion
Astatementinaprogramthataconditionistrueatthispointintheprogram.Usefulforreasoningabouthowa
programissupposedtobehave.
Assignment
Anawkexpressionthatchangesthevalueofsomeawkvariableordataobject.Anobjectthatyoucanassigntois
calledanlvalue.Theassignedvaluesarecalledrvalues.SeeAssignmentOps.
AssociativeArray
Arraysinwhichtheindicesmaybenumbersorstrings,notjustsequentialintegersinafixedrange.
awkLanguage
Thelanguageinwhichawkprogramsarewritten.
awkProgram
Anawkprogramconsistsofaseriesofpatternsandactions,collectivelyknownasrules.Foreachinputrecord
giventotheprogram,theprogramsrulesareallprocessedinturn.awkprogramsmayalsocontainfunction
definitions.
awkScript
Anothernameforanawkprogram.
Bash
TheGNUversionofthestandardshell(theBourneAgainSHell).SeealsoBourneShell.
Binary
Basetwonotation,wherethedigitsare01.Sinceelectroniccircuitryworksnaturallyinbase2(justthinkof
Off/On),everythinginsideacomputeriscalculatedusingbase2.Eachdigitrepresentsthepresence(orabsence)
ofapowerof2andiscalledabit.So,forexample,thebasetwonumber10101isthesameasdecimal21,((1x
16)+(1x4)+(1x1)).
Sincebasetwonumbersquicklybecomeverylongtoreadandwrite,theyareusuallygroupedby3(i.e.,theyare
readasoctalnumbers),orby4(i.e.,theyarereadashexadecimalnumbers).Thereisnodirectwaytoinsertbase
2numbersinaCprogram.Ifneedarises,suchnumbersareusuallyinsertedasoctalorhexadecimalnumbers.
Thenumberofbasetwodigitsthatfitintoregistersusedforrepresentingintegernumbersincomputersisarough
indicationofthecomputingpowerofthecomputeritself.Mostcomputersnowadaysuse64bitsforrepresenting
integernumbersintheirregisters,but32bit,16bitand8bitregistershavebeenwidelyusedinthepast.See
Nondecimalnumbers.
Bit
ShortforBinaryDigit.Allvaluesincomputermemoryultimatelyreducetobinarydigits:valuesthatareeither
zeroorone.Groupsofbitsmaybeinterpreteddifferentlyasintegers,floatingpointnumbers,characterdata,
addressesofothermemoryobjects,orotherdata.awkletsyouworkwithfloatingpointnumbersandstrings.gawk
letsyoumanipulatebitvalueswiththebuiltinfunctionsdescribedinBitwiseFunctions.
Computersareoftendefinedbyhowmanybitstheyusetorepresentintegervalues.Typicalsystemsare32bit
388/479
29/7/2016
TheGNUAwkUsersGuide
systems,but64bitsystemsarebecomingincreasinglypopular,and16bitsystemshaveessentiallydisappeared.
BooleanExpression
NamedaftertheEnglishmathematicianBoole.SeealsoLogicalExpression.
BourneShell
Thestandardshell(/bin/sh)onUnixandUnixlikesystems,originallywrittenbyStevenR.BourneatBell
Laboratories.Manyshells(Bash,ksh,pdksh,zsh)aregenerallyupwardlycompatiblewiththeBourneshell.
Braces
Thecharacters{and}.Bracesareusedinawkfordelimitingactions,compoundstatements,andfunction
bodies.
BracketExpression
Insidearegularexpression,anexpressionincludedinsquarebrackets,meanttodesignateasinglecharacteras
belongingtoaspecifiedcharacterclass.Abracketexpressioncancontainalistofoneormorecharacters,like
[abc],arangeofcharacters,like[AZ],oraname,delimitedby:,thatdesignatesaknownsetofcharacters,
like[:digit:].Theformofbracketexpressionenclosedbetween:isindependentoftheunderlying
representationofthecharacterthemselves,whichcouldutilizetheASCII,ECBDIC,orUnicodecodesets,
dependingonthearchitectureofthecomputersystem,andonlocalization.SeealsoRegularExpression.
BuiltinFunction
Theawklanguageprovidesbuiltinfunctionsthatperformvariousnumerical,I/Orelated,andstring
computations.Examplesaresqrt()(forthesquarerootofanumber)andsubstr()(forasubstringofastring).
gawkprovidesfunctionsfortimestampmanagement,bitmanipulation,arraysorting,typechecking,andruntime
stringtranslation.(SeeBuiltin.)
BuiltinVariable
ARGC,ARGV,CONVFMT,ENVIRON,FILENAME,FNR,FS,NF,NR,OFMT,OFS,ORS,RLENGTH,RSTART,RS,andSUBSEParethe
variablesthathavespecialmeaningtoawk.Inaddition,ARGIND,BINMODE,ERRNO,FIELDWIDTHS,FPAT,IGNORECASE,
LINT,PROCINFO,RT,andTEXTDOMAINarethevariablesthathavespecialmeaningtogawk.Changingsomeofthem
affectsawksrunningenvironment.(SeeBuiltinVariables.)
C
ThesystemprogramminglanguagethatmostGNUsoftwareiswrittenin.TheawkprogramminglanguagehasC
likesyntax,andthisWebpagepointsoutsimilaritiesbetweenawkandCwhenappropriate.
Ingeneral,gawkattemptstobeassimilartothe1990versionofISOCasmakessense.
CShell
TheCShell(cshoritsimprovedversion,tcsh)isaUnixshellthatwascreatedbyBillJoyinthelate1970s.The
Cshellwasdifferentiatedfromothershellsbyitsinteractivefeaturesandoverallstyle,whichlooksmorelikeC.
TheCShellisnotbackwardcompatiblewiththeBourneShell,sospecialattentionisrequiredwhenconverting
scriptswrittenforotherUnixshellstotheCshell,especiallywithregardtothemanagementofshellvariables.
SeealsoBourneShell.
C++
ApopularobjectorientedprogramminglanguagederivedfromC.
CharacterClass
389/479
29/7/2016
TheGNUAwkUsersGuide
SeeBracketExpression.
CharacterList
SeeBracketExpression.
CharacterSet
Thesetofnumericcodesusedbyacomputersystemtorepresentthecharacters(letters,numbers,punctuation,
etc.)ofaparticularcountryorplace.ThemostcommoncharactersetinusetodayisASCII(AmericanStandard
CodeforInformationInterchange).ManyEuropeancountriesuseanextensionofASCIIknownasISO88591
(ISOLatin1).TheUnicodecharactersetisincreasinglypopularandstandard,andisparticularlywidelyusedon
GNU/Linuxsystems.
CHEM
Apreprocessorforpicthatreadsdescriptionsofmoleculesandproducespicinputfordrawingthem.Itwas
writteninawkbyBrianKernighanandJonBentley,andisavailablefromhttp://netlib.org/typesetting/chem.
ComparisonExpression
Arelationthatiseithertrueorfalse,suchasa<b.Comparisonexpressionsareusedinif,while,do,andfor
statements,andinpatternstoselectwhichinputrecordstoprocess.(SeeTypingandComparison.)
Compiler
Aprogramthattranslateshumanreadablesourcecodeintomachineexecutableobjectcode.Theobjectcodeis
thenexecuteddirectlybythecomputer.SeealsoInterpreter.
ComplementedBracketExpression
Thenegationofabracketexpression.Allthatisnotdescribedbyagivenbracketexpression.Thesymbol^
precedesthenegatedbracketexpression.E.g.:[[^:digit:]designateswhatevercharacterisnotadigit. [^bad]
designateswhatevercharacterisnotoneofthelettersb,a,ord.SeeBracketExpression.
CompoundStatement
Aseriesofawkstatements,enclosedincurlybraces.Compoundstatementsmaybenested.(SeeStatements.)
ComputedRegexps
SeeDynamicRegularExpressions.
Concatenation
Concatenatingtwostringsmeansstickingthemtogether,oneafteranother,producinganewstring.Forexample,
thestringfooconcatenatedwiththestringbargivesthestringfoobar.(SeeConcatenation.)
ConditionalExpression
Anexpressionusingthe?:ternaryoperator,suchasexpr1?expr2:expr3.Theexpressionexpr1is
evaluatediftheresultistrue,thevalueofthewholeexpressionisthevalueofexpr2otherwisethevalueis
expr3.Ineithercase,onlyoneofexpr2andexpr3isevaluated.(SeeConditionalExp.)
ControlStatement
Acontrolstatementisaninstructiontoperformagivenoperationorasetofoperationsinsideanawkprogram,ifa
givenconditionistrue.Controlstatementsare:if,for,while,anddo(seeStatements).
Cookie
390/479
29/7/2016
TheGNUAwkUsersGuide
Apeculiargoodie,token,sayingorremembranceproducedbyorpresentedtoaprogram.(Withthanksto
ProfessorDougMcIlroy.)
Coprocess
Asubordinateprogramwithwhichtwowaycommunicationsispossible.
CurlyBraces
SeeBraces.
DarkCorner
Anareainthelanguagewherespecificationsoftenwere(orstillare)notclear,leadingtounexpectedor
undesirablebehavior.SuchareasaremarkedinthisWebpagewith(d.c.)inthetextandareindexedunderthe
headingdarkcorner.
DataDriven
Adescriptionofawkprograms,whereyouspecifythedatayouareinterestedinprocessing,andwhattodowhen
thatdataisseen.
DataObjects
Thesearenumbersandstringsofcharacters.Numbersareconvertedintostringsandviceversa,asneeded.(See
Conversion.)
Deadlock
Thesituationinwhichtwocommunicatingprocessesareeachwaitingfortheothertoperformanaction.
Debugger
Aprogramusedtohelpdevelopersremovebugsfrom(debug)theirprograms.
DoublePrecision
Aninternalrepresentationofnumbersthatcanhavefractionalparts.Doubleprecisionnumberskeeptrackof
moredigitsthandosingleprecisionnumbers,butoperationsonthemaresometimesmoreexpensive.Thisisthe
wayawkstoresnumericvalues.ItistheCtypedouble.
DynamicRegularExpression
Adynamicregularexpressionisaregularexpressionwrittenasanordinaryexpression.Itcouldbeastring
constant,suchas"foo",butitmayalsobeanexpressionwhosevaluecanvary.(SeeComputedRegexps.)
EmptyString
SeeNullString.
Environment
Acollectionofstrings,oftheformname=val,thateachprogramhasavailabletoit.Usersgenerallyplacevalues
intotheenvironmentinordertoprovideinformationtovariousprograms.Typicalexamplesaretheenvironment
variablesHOMEandPATH.
Epoch
Thedateusedasthebeginningoftimefortimestamps.Timevaluesinmostsystemsarerepresentedasseconds
sincetheepoch,withlibraryfunctionsavailableforconvertingthesevaluesintostandarddateandtimeformats.
391/479
29/7/2016
TheGNUAwkUsersGuide
TheepochonUnixandPOSIXsystemsis1970010100:00:00UTC.SeealsoGMTandUTC.
EscapeSequences
Aspecialsequenceofcharactersusedfordescribingnonprintingcharacters,suchas\nfornewlineor\033for
theASCIIESC(Escape)character.(SeeEscapeSequences.)
Extension
Anadditionalfeatureorchangetoaprogramminglanguageorutilitynotdefinedbythatlanguagesorutilitys
standard.gawkhas(too)manyextensionsoverPOSIXawk.
FDL
SeeFreeDocumentationLicense.
Field
Whenawkreadsaninputrecord,itsplitstherecordintopiecesseparatedbywhitespace(orbyaseparatorregexp
thatyoucanchangebysettingthepredefinedvariableFS).Suchpiecesarecalledfields.Ifthepiecesareoffixed
length,youcanusethebuiltinvariableFIELDWIDTHStodescribetheirlengths.Ifyouwishtospecifythecontents
offieldsinsteadofthefieldseparator,youcanusethepredefinedvariableFPATtodoso.(SeeFieldSeparators,
ConstantSize,andSplittingByContent.)
Flag
Avariablewhosetruthvalueindicatestheexistenceornonexistenceofsomecondition.
FloatingPointNumber
Oftenreferredtoinmathematicaltermsasarationalorrealnumber,thisisjustanumberthatcanhavea
fractionalpart.SeealsoDoublePrecisionandSinglePrecision.
Format
Formatstringscontroltheappearanceofoutputinthestrftime()andsprintf()functions,andintheprintf
statementaswell.Also,dataconversionsfromnumberstostringsarecontrolledbytheformatstringscontained
inthepredefinedvariablesCONVFMTandOFMT.(SeeControlLetters.)
Fortran
ShorthandforFORmulaTRANslator,oneofthefirstprogramminglanguagesavailableforscientificcalculations.
ItwascreatedbyJohnBackus,andhasbeenavailablesince1957.Itisstillinusetoday.
FreeDocumentationLicense
ThisdocumentdescribesthetermsunderwhichthisWebpageispublishedandmaybecopied.(SeeGNUFree
DocumentationLicense.)
FreeSoftwareFoundation
Anonprofitorganizationdedicatedtotheproductionanddistributionoffreelydistributablesoftware.Itwas
foundedbyRichardM.Stallman,theauthoroftheoriginalEmacseditor.GNUEmacsisthemostwidelyused
versionofEmacstoday.
FSF
SeeFreeSoftwareFoundation.
Function
392/479
29/7/2016
TheGNUAwkUsersGuide
Apartofanawkprogramthatcanbeinvokedfromeverypointoftheprogram,toperformatask.awkhasseveral
builtinfunctions.Userscandefinetheirownfunctionsineverypartoftheprogram.Functioncanberecursive,
i.e.,theymayinvokethemselves.SeeFunctions.Ingawkitisalsopossibletohavefunctionssharedamong
differentprograms,andincludedwhererequiredusingthe@includedirective(seeIncludeFiles).Ingawkthename
ofthefunctionthatshouldbeinvokedcanbegeneratedatruntime,i.e.,dynamically.ThegawkextensionAPI
providesconstructorfunctions(seeConstructorFunctions).
gawk
TheGNUimplementationofawk.
GeneralPublicLicense
Thisdocumentdescribesthetermsunderwhichgawkanditssourcecodemaybedistributed.(SeeCopying.)
GMT
GreenwichMeanTime.ThisistheoldtermforUTC.ItisthetimeofdayusedinternallyforUnixandPOSIX
systems.SeealsoEpochandUTC.
GNU
GNUsnotUnix.AnongoingprojectoftheFreeSoftwareFoundationtocreateacomplete,freely
distributable,POSIXcompliantcomputingenvironment.
GNU/Linux
AvariantoftheGNUsystemusingtheLinuxkernel,insteadoftheFreeSoftwareFoundationsHurdkernel.The
Linuxkernelisastable,efficient,fullfeaturedcloneofUnixthathasbeenportedtoavarietyofarchitectures.It
ismostpopularonPCclasssystems,butrunswellonavarietyofothersystemstoo.TheLinuxkernelsource
codeisavailableunderthetermsoftheGNUGeneralPublicLicense,whichisperhapsitsmostimportantaspect.
GPL
SeeGeneralPublicLicense.
Hexadecimal
Base16notation,wherethedigitsare09andAF,withArepresenting10, Brepresenting11,andsoon,upto
Ffor15.HexadecimalnumbersarewritteninCusingaleading0x,toindicatetheirbase.Thus,0x12is18((1
x16)+2).SeeNondecimalnumbers.
I/O
AbbreviationforInput/Output,theactofmovingdataintoand/oroutofarunningprogram.
InputRecord
Asinglechunkofdatathatisreadinbyawk.Usually,anawkinputrecordconsistsofonelineoftext.(See
Records.)
Integer
Awholenumber,i.e.,anumberthatdoesnothaveafractionalpart.
Internationalization
Theprocessofwritingormodifyingaprogramsothatitcanusemultiplelanguageswithoutrequiringfurther
sourcecodechanges.
393/479
29/7/2016
TheGNUAwkUsersGuide
Interpreter
Aprogramthatreadshumanreadablesourcecodedirectly,andusestheinstructionsinittoprocessdataand
produceresults.awkistypically(butnotalways)implementedasaninterpreter.SeealsoCompiler.
IntervalExpression
Acomponentofaregularexpressionthatletsyouspecifyrepeatedmatchesofsomepartoftheregexp.Interval
expressionswerenotoriginallyavailableinawkprograms.
ISO
TheInternationalOrganizationforStandardization.Thisorganizationproducesinternationalstandardsformany
things,includingprogramminglanguages,suchasCandC++.Inthecomputerarena,importantstandardslike
thoseforC,C++,andPOSIXbecomebothAmericannationalandISOinternationalstandardssimultaneously.
ThisWebpagereferstoStandardCasISOCthroughout.SeetheISOwebsiteformoreinformationaboutthe
nameoftheorganizationanditslanguageindependentthreeletteracronym.
Java
AmodernprogramminglanguageoriginallydevelopedbySunMicrosystems(nowOracle)supportingObject
Orientedprogramming.Althoughusuallyimplementedbycompilingtotheinstructionsforastandardvirtual
machine(theJVM),thelanguagecanbecompiledtonativecode.
Keyword
Intheawklanguage,akeywordisawordthathasspecialmeaning.Keywordsarereservedandmaynotbeusedas
variablenames.
gawkskeywordsare:BEGIN,BEGINFILE,END,ENDFILE,break,case,continue,defaultdelete,dowhile,else,exit,
forin,for,function,func,if,next,nextfile,switch,andwhile.
KornShell
TheKornShell(ksh)isaUnixshellwhichwasdevelopedbyDavidKornatBellLaboratoriesintheearly1980s.
TheKornShellisbackwardcompatiblewiththeBourneshellandincludesmanyfeaturesoftheCshell.Seealso
BourneShell.
LesserGeneralPublicLicense
Thisdocumentdescribesthetermsunderwhichbinarylibraryarchivesorsharedobjects,andtheirsourcecode
maybedistributed.
LGPL
SeeLesserGeneralPublicLicense.
Linux
SeeGNU/Linux.
Localization
Theprocessofprovidingthedatanecessaryforaninternationalizedprogramtoworkinaparticularlanguage.
LogicalExpression
Anexpressionusingtheoperatorsforlogic,AND,OR,andNOT,written&&,||,and!inawk.Oftencalled
Booleanexpressions,afterthemathematicianwhopioneeredthiskindofmathematicallogic.
394/479
29/7/2016
TheGNUAwkUsersGuide
Lvalue
Anexpressionthatcanappearontheleftsideofanassignmentoperator.Inmostlanguages,lvaluescanbe
variablesorarrayelements.Inawk,afielddesignatorcanalsobeusedasanlvalue.
Matching
Theactoftestingastringagainstaregularexpression.Iftheregexpdescribesthecontentsofthestring,itissaid
tomatchit.
Metacharacters
Charactersusedwithinaregexpthatdonotstandforthemselves.Instead,theydenoteregularexpression
operations,suchasrepetition,grouping,oralternation.
Nesting
Nestingiswhereinformationisorganizedinlayers,orwhereobjectscontainothersimilarobjects.Ingawkthe
@includedirectivecanbenested.Thenaturalnestingofarithmeticandlogicaloperationscanbechangedusing
parentheses(seePrecedence).
Noop
Anoperationthatdoesnothing.
NullString
Astringwithnocharactersinit.Itisrepresentedexplicitlyinawkprogramsbyplacingtwodoublequote
charactersnexttoeachother("").Itcanappearininputdatabyhavingtwosuccessiveoccurrencesofthefield
separatorappearnexttoeachother.
Number
Anumericvalueddataobject.Modernawkimplementationsusedoubleprecisionfloatingpointtorepresent
numbers.Ancientawkimplementationsusedsingleprecisionfloatingpoint.
Octal
Baseeightnotation,wherethedigitsare07.OctalnumbersarewritteninCusingaleading0,toindicatetheir
base.Thus,013is11((1x8)+3).SeeNondecimalnumbers.
OutputRecord
Asinglechunkofdatathatiswrittenoutbyawk.Usually,anawkoutputrecordconsistsofoneormorelinesof
text.SeeRecords.
Pattern
Patternstellawkwhichinputrecordsareinterestingtowhichrules.
Apatternisanarbitraryconditionalexpressionagainstwhichinputistested.Iftheconditionissatisfied,the
patternissaidtomatchtheinputrecord.Atypicalpatternmightcomparetheinputrecordagainstaregular
expression.(SeePatternOverview.)
PEBKAC
Anacronymdescribingwhatispossiblythemostfrequentsourceofcomputerusageproblems.(ProblemExists
BetweenKeyboardAndChair.)
Plugin
395/479
29/7/2016
TheGNUAwkUsersGuide
SeeExtensions.
POSIX
ThenameforaseriesofstandardsthatspecifyaPortableOperatingSysteminterface.TheIXdenotestheUnix
heritageofthesestandards.ThemainstandardofinterestforawkusersisIEEEStandardforInformation
Technology,Standard1003.12008.The2008POSIXstandardcanbefoundonlineat
http://www.opengroup.org/onlinepubs/9699919799/.
Precedence
Theorderinwhichoperationsareperformedwhenoperatorsareusedwithoutexplicitparentheses.
Private
Variablesand/orfunctionsthataremeantforuseexclusivelybylibraryfunctionsandnotforthemainawk
program.Specialcaremustbetakenwhennamingsuchvariablesandfunctions.(SeeLibraryNames.)
Range(ofinputlines)
Asequenceofconsecutivelinesfromtheinputfile(s).Apatterncanspecifyrangesofinputlinesforawkto
processoritcanspecifysinglelines.(SeePatternOverview.)
Record
SeeInputrecordandOutputrecord.
Recursion
Whenafunctioncallsitself,eitherdirectlyorindirectly.Ifthisisclear,stop,andproceedtothenextentry.
Otherwise,refertotheentryforrecursion.
Redirection
Redirectionmeansperforminginputfromsomethingotherthanthestandardinputstream,orperformingoutputto
somethingotherthanthestandardoutputstream.
Youcanredirectinputtothegetlinestatementusingthe<,|,and|&operators.Youcanredirecttheoutput
oftheprintandprintfstatementstoafileorasystemcommand,usingthe>,>>,|,and|&operators.(See
Getline,andRedirection.)
ReferenceCounts
Aninternalmechanismingawktominimizetheamountofmemoryneededtostorethevalueofstringvariables.If
thevalueassumedbyavariableisusedinmorethanoneplace,onlyonecopyofthevalueitselfiskept,andthe
associatedreferencecountisincreasedwhenthesamevalueisusedbyanadditionalvariable,anddecresedwhen
therelatedvariableisnolongerinuse.Whenthereferencecountgoestozero,thememoryspaceusedtostorethe
valueofthevariableisfreed.
Regexp
SeeRegularExpression.
RegularExpression
Aregularexpression(regexpforshort)isapatternthatdenotesasetofstrings,possiblyaninfiniteset.For
example,theregularexpressionR.*xpmatchesanystringstartingwiththeletterRandendingwiththeletters
xp.Inawk,regularexpressionsareusedinpatternsandinconditionalexpressions.Regularexpressionsmay
containescapesequences.(SeeRegexp.)
396/479
29/7/2016
TheGNUAwkUsersGuide
RegularExpressionConstant
Aregularexpressionconstantisaregularexpressionwrittenwithinslashes,suchas/foo/.Thisregular
expressionischosenwhenyouwritetheawkprogramandcannotbechangedduringitsexecution.(SeeRegexp
Usage.)
RegularExpressionOperators
SeeMetacharacters.
Rounding
Roundingtheresultofanarithmeticoperationcanbetricky.Morethanonewayofroundingexists,andingawkit
ispossibletochoosewhichmethodshouldbeusedinaprogram.SeeSettingtheroundingmode.
Rule
Asegmentofanawkprogramthatspecifieshowtoprocesssingleinputrecords.Aruleconsistsofapatternand
anaction.awkreadsaninputrecordthen,foreachrule,iftheinputrecordsatisfiestherulespattern,awkexecutes
therulesaction.Otherwise,theruledoesnothingforthatinputrecord.
Rvalue
Avaluethatcanappearontherightsideofanassignmentoperator.Inawk,essentiallyeveryexpressionhasa
value.Thesevaluesarervalues.
Scalar
Asinglevalue,beitanumberorastring.Regularvariablesarescalarsarraysandfunctionsarenot.
SearchPath
Ingawk,alistofdirectoriestosearchforawkprogramsourcefiles.Intheshell,alistofdirectoriestosearchfor
executableprograms.
sed
SeeStreamEditor.
Seed
Theinitialvalue,orstartingpoint,forasequenceofrandomnumbers.
Shell
ThecommandinterpreterforUnixandPOSIXcompliantsystems.Theshellworksbothinteractively,andasa
programminglanguageforbatchfiles,orshellscripts.
ShortCircuit
Thenatureoftheawklogicaloperators&&and||.Ifthevalueoftheentireexpressionisdeterminablefrom
evaluatingjustthelefthandsideoftheseoperators,therighthandsideisnotevaluated.(SeeBooleanOps.)
SideEffect
Asideeffectoccurswhenanexpressionhasaneffectasidefrommerelyproducingavalue.Assignment
expressions,incrementanddecrementexpressions,andfunctioncallshavesideeffects.(SeeAssignmentOps.)
SinglePrecision
397/479
29/7/2016
TheGNUAwkUsersGuide
Aninternalrepresentationofnumbersthatcanhavefractionalparts.Singleprecisionnumberskeeptrackoffewer
digitsthandodoubleprecisionnumbers,butoperationsonthemaresometimeslessexpensiveintermsofCPU
time.Thisisthetypeusedbysomeancientversionsofawktostorenumericvalues.ItistheCtypefloat.
Space
Thecharactergeneratedbyhittingthespacebaronthekeyboard.
SpecialFile
Afilenameinterpretedinternallybygawk,insteadofbeinghandeddirectlytotheunderlyingoperatingsystem
forexample,/dev/stderr.(SeeSpecialFiles.)
Statement
Anexpressioninsideanawkprogramintheactionpartofapatternactionrule,orinsideanawkfunction.A
statementcanbeavariableassignment,anarrayoperation,aloop,etc.
StreamEditor
Aprogramthatreadsrecordsfromaninputstreamandprocessesthemoneormoreatatime.Thisisincontrast
withbatchprograms,whichmayexpecttoreadtheirinputfilesinentiretybeforestartingtodoanything,aswell
aswithinteractiveprogramswhichrequireinputfromtheuser.
String
Adatumconsistingofasequenceofcharacters,suchasIamastring.Constantstringsarewrittenwithdouble
quotesintheawklanguageandmaycontainescapesequences.(SeeEscapeSequences.)
Tab
ThecharactergeneratedbyhittingtheTABkeyonthekeyboard.Itusuallyexpandstouptoeightspacesupon
output.
TextDomain
Auniquenamethatidentifiesanapplication.Usedforgroupingmessagesthataretranslatedatruntimeintothe
locallanguage.
Timestamp
AvalueinthesecondssincetheepochformatusedbyUnixandPOSIXsystems.Usedforthegawkfunctions
mktime(),strftime(),andsystime().SeealsoEpoch,GMT,andUTC.
Unix
Acomputeroperatingsystemoriginallydevelopedintheearly1970satAT&TBellLaboratories.Itinitially
becamepopularinuniversitiesaroundtheworldandlatermovedintocommercialenvironmentsasasoftware
developmentsystemandnetworkserversystem.TherearemanycommercialversionsofUnix,aswellasseveral
workalikesystemswhosesourcecodeisfreelyavailable(suchasGNU/Linux,NetBSD,FreeBSD,and
OpenBSD).
UTC
TheacceptedabbreviationforUniversalCoordinatedTime.ThisisstandardtimeinGreenwich,England,
whichisusedasareferencetimefordayanddatecalculations.SeealsoEpochandGMT.
Variable
Anameforavalue.Inawk,variablesmaybeeitherscalarsorarrays.
398/479
29/7/2016
TheGNUAwkUsersGuide
Whitespace
Asequenceofspace,TAB,ornewlinecharactersoccurringinsideaninputrecordorastring.
Next:GNUFreeDocumentationLicense,Previous:Glossary,Up:Top[Contents][Index]
Version3,29June2007
Copyright2007FreeSoftwareFoundation,Inc.http://fsf.org/
Everyoneispermittedtocopyanddistributeverbatimcopiesofthis
licensedocument,butchangingitisnotallowed.
Preamble
TheGNUGeneralPublicLicenseisafree,copyleftlicenseforsoftwareandotherkindsofworks.
Thelicensesformostsoftwareandotherpracticalworksaredesignedtotakeawayyourfreedomtoshareandchange
theworks.Bycontrast,theGNUGeneralPublicLicenseisintendedtoguaranteeyourfreedomtoshareandchangeall
versionsofaprogramtomakesureitremainsfreesoftwareforallitsusers.We,theFreeSoftwareFoundation,use
theGNUGeneralPublicLicenseformostofoursoftwareitappliesalsotoanyotherworkreleasedthiswaybyits
authors.Youcanapplyittoyourprograms,too.
Whenwespeakoffreesoftware,wearereferringtofreedom,notprice.OurGeneralPublicLicensesaredesignedto
makesurethatyouhavethefreedomtodistributecopiesoffreesoftware(andchargeforthemifyouwish),thatyou
receivesourcecodeorcangetitifyouwantit,thatyoucanchangethesoftwareorusepiecesofitinnewfree
programs,andthatyouknowyoucandothesethings.
Toprotectyourrights,weneedtopreventothersfromdenyingyoutheserightsoraskingyoutosurrendertherights.
Therefore,youhavecertainresponsibilitiesifyoudistributecopiesofthesoftware,orifyoumodifyit:responsibilities
torespectthefreedomofothers.
Forexample,ifyoudistributecopiesofsuchaprogram,whethergratisorforafee,youmustpassontotherecipients
thesamefreedomsthatyoureceived.Youmustmakesurethatthey,too,receiveorcangetthesourcecode.Andyou
mustshowthemthesetermssotheyknowtheirrights.
DevelopersthatusetheGNUGPLprotectyourrightswithtwosteps:(1)assertcopyrightonthesoftware,and(2)offer
youthisLicensegivingyoulegalpermissiontocopy,distributeand/ormodifyit.
Forthedevelopersandauthorsprotection,theGPLclearlyexplainsthatthereisnowarrantyforthisfreesoftware.For
bothusersandauthorssake,theGPLrequiresthatmodifiedversionsbemarkedaschanged,sothattheirproblems
willnotbeattributederroneouslytoauthorsofpreviousversions.
Somedevicesaredesignedtodenyusersaccesstoinstallorrunmodifiedversionsofthesoftwareinsidethem,although
themanufacturercandoso.Thisisfundamentallyincompatiblewiththeaimofprotectingusersfreedomtochangethe
software.Thesystematicpatternofsuchabuseoccursintheareaofproductsforindividualstouse,whichisprecisely
whereitismostunacceptable.Therefore,wehavedesignedthisversionoftheGPLtoprohibitthepracticeforthose
products.Ifsuchproblemsarisesubstantiallyinotherdomains,westandreadytoextendthisprovisiontothose
domainsinfutureversionsoftheGPL,asneededtoprotectthefreedomofusers.
Finally,everyprogramisthreatenedconstantlybysoftwarepatents.Statesshouldnotallowpatentstorestrict
developmentanduseofsoftwareongeneralpurposecomputers,butinthosethatdo,wewishtoavoidthespecial
dangerthatpatentsappliedtoafreeprogramcouldmakeiteffectivelyproprietary.Topreventthis,theGPLassuresthat
patentscannotbeusedtorendertheprogramnonfree.
399/479
29/7/2016
TheGNUAwkUsersGuide
Theprecisetermsandconditionsforcopying,distributionandmodificationfollow.
TERMSANDCONDITIONS
1.Definitions.
ThisLicensereferstoversion3oftheGNUGeneralPublicLicense.
Copyrightalsomeanscopyrightlikelawsthatapplytootherkindsofworks,suchassemiconductormasks.
TheProgramreferstoanycopyrightableworklicensedunderthisLicense.Eachlicenseeisaddressedasyou.
Licenseesandrecipientsmaybeindividualsororganizations.
Tomodifyaworkmeanstocopyfromoradaptallorpartoftheworkinafashionrequiringcopyright
permission,otherthanthemakingofanexactcopy.Theresultingworkiscalledamodifiedversionofthe
earlierworkoraworkbasedontheearlierwork.
AcoveredworkmeanseithertheunmodifiedProgramoraworkbasedontheProgram.
Topropagateaworkmeanstodoanythingwithitthat,withoutpermission,wouldmakeyoudirectlyor
secondarilyliableforinfringementunderapplicablecopyrightlaw,exceptexecutingitonacomputeror
modifyingaprivatecopy.Propagationincludescopying,distribution(withorwithoutmodification),making
availabletothepublic,andinsomecountriesotheractivitiesaswell.
Toconveyaworkmeansanykindofpropagationthatenablesotherpartiestomakeorreceivecopies.Mere
interactionwithauserthroughacomputernetwork,withnotransferofacopy,isnotconveying.
AninteractiveuserinterfacedisplaysAppropriateLegalNoticestotheextentthatitincludesaconvenientand
prominentlyvisiblefeaturethat(1)displaysanappropriatecopyrightnotice,and(2)tellstheuserthatthereisno
warrantyforthework(excepttotheextentthatwarrantiesareprovided),thatlicenseesmayconveythework
underthisLicense,andhowtoviewacopyofthisLicense.Iftheinterfacepresentsalistofusercommandsor
options,suchasamenu,aprominentiteminthelistmeetsthiscriterion.
2.SourceCode.
Thesourcecodeforaworkmeansthepreferredformoftheworkformakingmodificationstoit.Objectcode
meansanynonsourceformofawork.
AStandardInterfacemeansaninterfacethateitherisanofficialstandarddefinedbyarecognizedstandards
body,or,inthecaseofinterfacesspecifiedforaparticularprogramminglanguage,onethatiswidelyusedamong
developersworkinginthatlanguage.
TheSystemLibrariesofanexecutableworkincludeanything,otherthantheworkasawhole,that(a)is
includedinthenormalformofpackagingaMajorComponent,butwhichisnotpartofthatMajorComponent,
and(b)servesonlytoenableuseoftheworkwiththatMajorComponent,ortoimplementaStandardInterface
forwhichanimplementationisavailabletothepublicinsourcecodeform.AMajorComponent,inthis
context,meansamajoressentialcomponent(kernel,windowsystem,andsoon)ofthespecificoperatingsystem
(ifany)onwhichtheexecutableworkruns,oracompilerusedtoproducethework,oranobjectcodeinterpreter
usedtorunit.
TheCorrespondingSourceforaworkinobjectcodeformmeansallthesourcecodeneededtogenerate,install,
and(foranexecutablework)runtheobjectcodeandtomodifythework,includingscriptstocontrolthose
activities.However,itdoesnotincludetheworksSystemLibraries,orgeneralpurposetoolsorgenerally
availablefreeprogramswhichareusedunmodifiedinperformingthoseactivitiesbutwhicharenotpartofthe
work.Forexample,CorrespondingSourceincludesinterfacedefinitionfilesassociatedwithsourcefilesforthe
work,andthesourcecodeforsharedlibrariesanddynamicallylinkedsubprogramsthattheworkisspecifically
designedtorequire,suchasbyintimatedatacommunicationorcontrolflowbetweenthosesubprogramsand
otherpartsofthework.
400/479
29/7/2016
TheGNUAwkUsersGuide
TheCorrespondingSourceneednotincludeanythingthatuserscanregenerateautomaticallyfromotherpartsof
theCorrespondingSource.
TheCorrespondingSourceforaworkinsourcecodeformisthatsamework.
3.BasicPermissions.
AllrightsgrantedunderthisLicensearegrantedforthetermofcopyrightontheProgram,andareirrevocable
providedthestatedconditionsaremet.ThisLicenseexplicitlyaffirmsyourunlimitedpermissiontorunthe
unmodifiedProgram.TheoutputfromrunningacoveredworkiscoveredbythisLicenseonlyiftheoutput,given
itscontent,constitutesacoveredwork.ThisLicenseacknowledgesyourrightsoffairuseorotherequivalent,as
providedbycopyrightlaw.
Youmaymake,runandpropagatecoveredworksthatyoudonotconvey,withoutconditionssolongasyour
licenseotherwiseremainsinforce.Youmayconveycoveredworkstoothersforthesolepurposeofhavingthem
makemodificationsexclusivelyforyou,orprovideyouwithfacilitiesforrunningthoseworks,providedthatyou
complywiththetermsofthisLicenseinconveyingallmaterialforwhichyoudonotcontrolcopyright.Those
thusmakingorrunningthecoveredworksforyoumustdosoexclusivelyonyourbehalf,underyourdirection
andcontrol,ontermsthatprohibitthemfrommakinganycopiesofyourcopyrightedmaterialoutsidetheir
relationshipwithyou.
Conveyingunderanyothercircumstancesispermittedsolelyundertheconditionsstatedbelow.Sublicensingis
notallowedsection10makesitunnecessary.
4.ProtectingUsersLegalRightsFromAntiCircumventionLaw.
Nocoveredworkshallbedeemedpartofaneffectivetechnologicalmeasureunderanyapplicablelawfulfilling
obligationsunderarticle11oftheWIPOcopyrighttreatyadoptedon20December1996,orsimilarlaws
prohibitingorrestrictingcircumventionofsuchmeasures.
Whenyouconveyacoveredwork,youwaiveanylegalpowertoforbidcircumventionoftechnologicalmeasures
totheextentsuchcircumventioniseffectedbyexercisingrightsunderthisLicensewithrespecttothecovered
work,andyoudisclaimanyintentiontolimitoperationormodificationoftheworkasameansofenforcing,
againsttheworksusers,yourorthirdpartieslegalrightstoforbidcircumventionoftechnologicalmeasures.
5.ConveyingVerbatimCopies.
YoumayconveyverbatimcopiesoftheProgramssourcecodeasyoureceiveit,inanymedium,providedthat
youconspicuouslyandappropriatelypublishoneachcopyanappropriatecopyrightnoticekeepintactallnotices
statingthatthisLicenseandanynonpermissivetermsaddedinaccordwithsection7applytothecodekeep
intactallnoticesoftheabsenceofanywarrantyandgiveallrecipientsacopyofthisLicensealongwiththe
Program.
Youmaychargeanypriceornopriceforeachcopythatyouconvey,andyoumayoffersupportorwarranty
protectionforafee.
6.ConveyingModifiedSourceVersions.
YoumayconveyaworkbasedontheProgram,orthemodificationstoproduceitfromtheProgram,intheform
ofsourcecodeunderthetermsofsection4,providedthatyoualsomeetalloftheseconditions:
1.Theworkmustcarryprominentnoticesstatingthatyoumodifiedit,andgivingarelevantdate.
2.TheworkmustcarryprominentnoticesstatingthatitisreleasedunderthisLicenseandanyconditions
addedundersection7.Thisrequirementmodifiestherequirementinsection4tokeepintactallnotices.
3.Youmustlicensetheentirework,asawhole,underthisLicensetoanyonewhocomesintopossessionofa
copy.ThisLicensewillthereforeapply,alongwithanyapplicablesection7additionalterms,tothewhole
ofthework,andallitsparts,regardlessofhowtheyarepackaged.ThisLicensegivesnopermissionto
licensetheworkinanyotherway,butitdoesnotinvalidatesuchpermissionifyouhaveseparatelyreceived
it.
401/479
29/7/2016
TheGNUAwkUsersGuide
4.Iftheworkhasinteractiveuserinterfaces,eachmustdisplayAppropriateLegalNoticeshowever,ifthe
ProgramhasinteractiveinterfacesthatdonotdisplayAppropriateLegalNotices,yourworkneednotmake
themdoso.
Acompilationofacoveredworkwithotherseparateandindependentworks,whicharenotbytheirnature
extensionsofthecoveredwork,andwhicharenotcombinedwithitsuchastoformalargerprogram,inorona
volumeofastorageordistributionmedium,iscalledanaggregateifthecompilationanditsresultingcopyright
arenotusedtolimittheaccessorlegalrightsofthecompilationsusersbeyondwhattheindividualworkspermit.
InclusionofacoveredworkinanaggregatedoesnotcausethisLicensetoapplytotheotherpartsofthe
aggregate.
7.ConveyingNonSourceForms.
Youmayconveyacoveredworkinobjectcodeformunderthetermsofsections4and5,providedthatyoualso
conveythemachinereadableCorrespondingSourceunderthetermsofthisLicense,inoneoftheseways:
1.Conveytheobjectcodein,orembodiedin,aphysicalproduct(includingaphysicaldistributionmedium),
accompaniedbytheCorrespondingSourcefixedonadurablephysicalmediumcustomarilyusedfor
softwareinterchange.
2.Conveytheobjectcodein,orembodiedin,aphysicalproduct(includingaphysicaldistributionmedium),
accompaniedbyawrittenoffer,validforatleastthreeyearsandvalidforaslongasyouoffersparepartsor
customersupportforthatproductmodel,togiveanyonewhopossessestheobjectcodeeither(1)acopyof
theCorrespondingSourceforallthesoftwareintheproductthatiscoveredbythisLicense,onadurable
physicalmediumcustomarilyusedforsoftwareinterchange,forapricenomorethanyourreasonablecost
ofphysicallyperformingthisconveyingofsource,or(2)accesstocopytheCorrespondingSourcefroma
networkserveratnocharge.
3.ConveyindividualcopiesoftheobjectcodewithacopyofthewrittenoffertoprovidetheCorresponding
Source.Thisalternativeisallowedonlyoccasionallyandnoncommercially,andonlyifyoureceivedthe
objectcodewithsuchanoffer,inaccordwithsubsection6b.
4.Conveytheobjectcodebyofferingaccessfromadesignatedplace(gratisorforacharge),andoffer
equivalentaccesstotheCorrespondingSourceinthesamewaythroughthesameplaceatnofurthercharge.
YouneednotrequirerecipientstocopytheCorrespondingSourcealongwiththeobjectcode.Iftheplaceto
copytheobjectcodeisanetworkserver,theCorrespondingSourcemaybeonadifferentserver(operated
byyouorathirdparty)thatsupportsequivalentcopyingfacilities,providedyoumaintaincleardirections
nexttotheobjectcodesayingwheretofindtheCorrespondingSource.Regardlessofwhatserverhoststhe
CorrespondingSource,youremainobligatedtoensurethatitisavailableforaslongasneededtosatisfy
theserequirements.
5.Conveytheobjectcodeusingpeertopeertransmission,providedyouinformotherpeerswheretheobject
codeandCorrespondingSourceoftheworkarebeingofferedtothegeneralpublicatnochargeunder
subsection6d.
Aseparableportionoftheobjectcode,whosesourcecodeisexcludedfromtheCorrespondingSourceasa
SystemLibrary,neednotbeincludedinconveyingtheobjectcodework.
AUserProductiseither(1)aconsumerproduct,whichmeansanytangiblepersonalpropertywhichis
normallyusedforpersonal,family,orhouseholdpurposes,or(2)anythingdesignedorsoldforincorporationinto
adwelling.Indeterminingwhetheraproductisaconsumerproduct,doubtfulcasesshallberesolvedinfavorof
coverage.Foraparticularproductreceivedbyaparticularuser,normallyusedreferstoatypicalorcommon
useofthatclassofproduct,regardlessofthestatusoftheparticularuserorofthewayinwhichtheparticularuser
actuallyuses,orexpectsorisexpectedtouse,theproduct.Aproductisaconsumerproductregardlessofwhether
theproducthassubstantialcommercial,industrialornonconsumeruses,unlesssuchusesrepresenttheonly
significantmodeofuseoftheproduct.
InstallationInformationforaUserProductmeansanymethods,procedures,authorizationkeys,orother
informationrequiredtoinstallandexecutemodifiedversionsofacoveredworkinthatUserProductfroma
modifiedversionofitsCorrespondingSource.Theinformationmustsufficetoensurethatthecontinued
functioningofthemodifiedobjectcodeisinnocasepreventedorinterferedwithsolelybecausemodificationhas
beenmade.
402/479
29/7/2016
TheGNUAwkUsersGuide
Ifyouconveyanobjectcodeworkunderthissectionin,orwith,orspecificallyforusein,aUserProduct,andthe
conveyingoccursaspartofatransactioninwhichtherightofpossessionanduseoftheUserProductis
transferredtotherecipientinperpetuityorforafixedterm(regardlessofhowthetransactionischaracterized),
theCorrespondingSourceconveyedunderthissectionmustbeaccompaniedbytheInstallationInformation.But
thisrequirementdoesnotapplyifneitheryounoranythirdpartyretainstheabilitytoinstallmodifiedobjectcode
ontheUserProduct(forexample,theworkhasbeeninstalledinROM).
TherequirementtoprovideInstallationInformationdoesnotincludearequirementtocontinuetoprovidesupport
service,warranty,orupdatesforaworkthathasbeenmodifiedorinstalledbytherecipient,orfortheUser
Productinwhichithasbeenmodifiedorinstalled.Accesstoanetworkmaybedeniedwhenthemodification
itselfmateriallyandadverselyaffectstheoperationofthenetworkorviolatestherulesandprotocolsfor
communicationacrossthenetwork.
CorrespondingSourceconveyed,andInstallationInformationprovided,inaccordwiththissectionmustbeina
formatthatispubliclydocumented(andwithanimplementationavailabletothepublicinsourcecodeform),and
mustrequirenospecialpasswordorkeyforunpacking,readingorcopying.
8.AdditionalTerms.
AdditionalpermissionsaretermsthatsupplementthetermsofthisLicensebymakingexceptionsfromoneor
moreofitsconditions.AdditionalpermissionsthatareapplicabletotheentireProgramshallbetreatedasthough
theywereincludedinthisLicense,totheextentthattheyarevalidunderapplicablelaw.Ifadditionalpermissions
applyonlytopartoftheProgram,thatpartmaybeusedseparatelyunderthosepermissions,buttheentire
ProgramremainsgovernedbythisLicensewithoutregardtotheadditionalpermissions.
Whenyouconveyacopyofacoveredwork,youmayatyouroptionremoveanyadditionalpermissionsfromthat
copy,orfromanypartofit.(Additionalpermissionsmaybewrittentorequiretheirownremovalincertaincases
whenyoumodifythework.)Youmayplaceadditionalpermissionsonmaterial,addedbyyoutoacoveredwork,
forwhichyouhaveorcangiveappropriatecopyrightpermission.
NotwithstandinganyotherprovisionofthisLicense,formaterialyouaddtoacoveredwork,youmay(if
authorizedbythecopyrightholdersofthatmaterial)supplementthetermsofthisLicensewithterms:
1.Disclaimingwarrantyorlimitingliabilitydifferentlyfromthetermsofsections15and16ofthisLicense
or
2.Requiringpreservationofspecifiedreasonablelegalnoticesorauthorattributionsinthatmaterialorinthe
AppropriateLegalNoticesdisplayedbyworkscontainingitor
3.Prohibitingmisrepresentationoftheoriginofthatmaterial,orrequiringthatmodifiedversionsofsuch
materialbemarkedinreasonablewaysasdifferentfromtheoriginalversionor
4.Limitingtheuseforpublicitypurposesofnamesoflicensorsorauthorsofthematerialor
5.Decliningtograntrightsundertrademarklawforuseofsometradenames,trademarks,orservicemarksor
6.Requiringindemnificationoflicensorsandauthorsofthatmaterialbyanyonewhoconveysthematerial(or
modifiedversionsofit)withcontractualassumptionsofliabilitytotherecipient,foranyliabilitythatthese
contractualassumptionsdirectlyimposeonthoselicensorsandauthors.
Allothernonpermissiveadditionaltermsareconsideredfurtherrestrictionswithinthemeaningofsection10.
IftheProgramasyoureceivedit,oranypartofit,containsanoticestatingthatitisgovernedbythisLicense
alongwithatermthatisafurtherrestriction,youmayremovethatterm.Ifalicensedocumentcontainsafurther
restrictionbutpermitsrelicensingorconveyingunderthisLicense,youmayaddtoacoveredworkmaterial
governedbythetermsofthatlicensedocument,providedthatthefurtherrestrictiondoesnotsurvivesuch
relicensingorconveying.
Ifyouaddtermstoacoveredworkinaccordwiththissection,youmustplace,intherelevantsourcefiles,a
statementoftheadditionaltermsthatapplytothosefiles,oranoticeindicatingwheretofindtheapplicable
terms.
Additionalterms,permissiveornonpermissive,maybestatedintheformofaseparatelywrittenlicense,or
statedasexceptionstheaboverequirementsapplyeitherway.
403/479
29/7/2016
TheGNUAwkUsersGuide
9.Termination.
YoumaynotpropagateormodifyacoveredworkexceptasexpresslyprovidedunderthisLicense.Anyattempt
otherwisetopropagateormodifyitisvoid,andwillautomaticallyterminateyourrightsunderthisLicense
(includinganypatentlicensesgrantedunderthethirdparagraphofsection11).
However,ifyouceaseallviolationofthisLicense,thenyourlicensefromaparticularcopyrightholderis
reinstated(a)provisionally,unlessanduntilthecopyrightholderexplicitlyandfinallyterminatesyourlicense,
and(b)permanently,ifthecopyrightholderfailstonotifyyouoftheviolationbysomereasonablemeanspriorto
60daysafterthecessation.
Moreover,yourlicensefromaparticularcopyrightholderisreinstatedpermanentlyifthecopyrightholder
notifiesyouoftheviolationbysomereasonablemeans,thisisthefirsttimeyouhavereceivednoticeofviolation
ofthisLicense(foranywork)fromthatcopyrightholder,andyoucuretheviolationpriorto30daysafteryour
receiptofthenotice.
Terminationofyourrightsunderthissectiondoesnotterminatethelicensesofpartieswhohavereceivedcopies
orrightsfromyouunderthisLicense.Ifyourrightshavebeenterminatedandnotpermanentlyreinstated,youdo
notqualifytoreceivenewlicensesforthesamematerialundersection10.
10.AcceptanceNotRequiredforHavingCopies.
YouarenotrequiredtoacceptthisLicenseinordertoreceiveorrunacopyoftheProgram.Ancillarypropagation
ofacoveredworkoccurringsolelyasaconsequenceofusingpeertopeertransmissiontoreceiveacopylikewise
doesnotrequireacceptance.However,nothingotherthanthisLicensegrantsyoupermissiontopropagateor
modifyanycoveredwork.TheseactionsinfringecopyrightifyoudonotacceptthisLicense.Therefore,by
modifyingorpropagatingacoveredwork,youindicateyouracceptanceofthisLicensetodoso.
11.AutomaticLicensingofDownstreamRecipients.
Eachtimeyouconveyacoveredwork,therecipientautomaticallyreceivesalicensefromtheoriginallicensors,
torun,modifyandpropagatethatwork,subjecttothisLicense.Youarenotresponsibleforenforcingcompliance
bythirdpartieswiththisLicense.
Anentitytransactionisatransactiontransferringcontrolofanorganization,orsubstantiallyallassetsofone,or
subdividinganorganization,ormergingorganizations.Ifpropagationofacoveredworkresultsfromanentity
transaction,eachpartytothattransactionwhoreceivesacopyoftheworkalsoreceiveswhateverlicensestothe
workthepartyspredecessorininteresthadorcouldgiveunderthepreviousparagraph,plusarighttopossession
oftheCorrespondingSourceoftheworkfromthepredecessorininterest,ifthepredecessorhasitorcangetit
withreasonableefforts.
YoumaynotimposeanyfurtherrestrictionsontheexerciseoftherightsgrantedoraffirmedunderthisLicense.
Forexample,youmaynotimposealicensefee,royalty,orotherchargeforexerciseofrightsgrantedunderthis
License,andyoumaynotinitiatelitigation(includingacrossclaimorcounterclaiminalawsuit)allegingthat
anypatentclaimisinfringedbymaking,using,selling,offeringforsale,orimportingtheProgramoranyportion
ofit.
12.Patents.
AcontributorisacopyrightholderwhoauthorizesuseunderthisLicenseoftheProgramoraworkonwhich
theProgramisbased.Theworkthuslicensediscalledthecontributorscontributorversion.
Acontributorsessentialpatentclaimsareallpatentclaimsownedorcontrolledbythecontributor,whether
alreadyacquiredorhereafteracquired,thatwouldbeinfringedbysomemanner,permittedbythisLicense,of
making,using,orsellingitscontributorversion,butdonotincludeclaimsthatwouldbeinfringedonlyasa
consequenceoffurthermodificationofthecontributorversion.Forpurposesofthisdefinition,controlincludes
therighttograntpatentsublicensesinamannerconsistentwiththerequirementsofthisLicense.
Eachcontributorgrantsyouanonexclusive,worldwide,royaltyfreepatentlicenseunderthecontributors
404/479
29/7/2016
TheGNUAwkUsersGuide
essentialpatentclaims,tomake,use,sell,offerforsale,importandotherwiserun,modifyandpropagatethe
contentsofitscontributorversion.
Inthefollowingthreeparagraphs,apatentlicenseisanyexpressagreementorcommitment,however
denominated,nottoenforceapatent(suchasanexpresspermissiontopracticeapatentorcovenantnottosuefor
patentinfringement).Tograntsuchapatentlicensetoapartymeanstomakesuchanagreementor
commitmentnottoenforceapatentagainsttheparty.
Ifyouconveyacoveredwork,knowinglyrelyingonapatentlicense,andtheCorrespondingSourceofthework
isnotavailableforanyonetocopy,freeofchargeandunderthetermsofthisLicense,throughapublicly
availablenetworkserverorotherreadilyaccessiblemeans,thenyoumusteither(1)causetheCorresponding
Sourcetobesoavailable,or(2)arrangetodepriveyourselfofthebenefitofthepatentlicenseforthisparticular
work,or(3)arrange,inamannerconsistentwiththerequirementsofthisLicense,toextendthepatentlicenseto
downstreamrecipients.Knowinglyrelyingmeansyouhaveactualknowledgethat,butforthepatentlicense,
yourconveyingthecoveredworkinacountry,oryourrecipientsuseofthecoveredworkinacountry,would
infringeoneormoreidentifiablepatentsinthatcountrythatyouhavereasontobelievearevalid.
If,pursuanttoorinconnectionwithasingletransactionorarrangement,youconvey,orpropagatebyprocuring
conveyanceof,acoveredwork,andgrantapatentlicensetosomeofthepartiesreceivingthecoveredwork
authorizingthemtouse,propagate,modifyorconveyaspecificcopyofthecoveredwork,thenthepatentlicense
yougrantisautomaticallyextendedtoallrecipientsofthecoveredworkandworksbasedonit.
Apatentlicenseisdiscriminatoryifitdoesnotincludewithinthescopeofitscoverage,prohibitstheexercise
of,orisconditionedonthenonexerciseofoneormoreoftherightsthatarespecificallygrantedunderthis
License.Youmaynotconveyacoveredworkifyouareapartytoanarrangementwithathirdpartythatisinthe
businessofdistributingsoftware,underwhichyoumakepaymenttothethirdpartybasedontheextentofyour
activityofconveyingthework,andunderwhichthethirdpartygrants,toanyofthepartieswhowouldreceive
thecoveredworkfromyou,adiscriminatorypatentlicense(a)inconnectionwithcopiesofthecoveredwork
conveyedbyyou(orcopiesmadefromthosecopies),or(b)primarilyforandinconnectionwithspecificproducts
orcompilationsthatcontainthecoveredwork,unlessyouenteredintothatarrangement,orthatpatentlicensewas
granted,priorto28March2007.
NothinginthisLicenseshallbeconstruedasexcludingorlimitinganyimpliedlicenseorotherdefensesto
infringementthatmayotherwisebeavailabletoyouunderapplicablepatentlaw.
13.NoSurrenderofOthersFreedom.
Ifconditionsareimposedonyou(whetherbycourtorder,agreementorotherwise)thatcontradicttheconditions
ofthisLicense,theydonotexcuseyoufromtheconditionsofthisLicense.Ifyoucannotconveyacoveredwork
soastosatisfysimultaneouslyyourobligationsunderthisLicenseandanyotherpertinentobligations,thenasa
consequenceyoumaynotconveyitatall.Forexample,ifyouagreetotermsthatobligateyoutocollectaroyalty
forfurtherconveyingfromthosetowhomyouconveytheProgram,theonlywayyoucouldsatisfyboththose
termsandthisLicensewouldbetorefrainentirelyfromconveyingtheProgram.
14.UsewiththeGNUAfferoGeneralPublicLicense.
NotwithstandinganyotherprovisionofthisLicense,youhavepermissiontolinkorcombineanycoveredwork
withaworklicensedunderversion3oftheGNUAfferoGeneralPublicLicenseintoasinglecombinedwork,
andtoconveytheresultingwork.ThetermsofthisLicensewillcontinuetoapplytothepartwhichisthecovered
work,butthespecialrequirementsoftheGNUAfferoGeneralPublicLicense,section13,concerninginteraction
throughanetworkwillapplytothecombinationassuch.
15.RevisedVersionsofthisLicense.
TheFreeSoftwareFoundationmaypublishrevisedand/ornewversionsoftheGNUGeneralPublicLicensefrom
timetotime.Suchnewversionswillbesimilarinspirittothepresentversion,butmaydifferindetailtoaddress
newproblemsorconcerns.
Eachversionisgivenadistinguishingversionnumber.IftheProgramspecifiesthatacertainnumberedversionof
405/479
29/7/2016
TheGNUAwkUsersGuide
theGNUGeneralPublicLicenseoranylaterversionappliestoit,youhavetheoptionoffollowingtheterms
andconditionseitherofthatnumberedversionorofanylaterversionpublishedbytheFreeSoftwareFoundation.
IftheProgramdoesnotspecifyaversionnumberoftheGNUGeneralPublicLicense,youmaychooseany
versioneverpublishedbytheFreeSoftwareFoundation.
IftheProgramspecifiesthataproxycandecidewhichfutureversionsoftheGNUGeneralPublicLicensecanbe
used,thatproxyspublicstatementofacceptanceofaversionpermanentlyauthorizesyoutochoosethatversion
fortheProgram.
Laterlicenseversionsmaygiveyouadditionalordifferentpermissions.However,noadditionalobligationsare
imposedonanyauthororcopyrightholderasaresultofyourchoosingtofollowalaterversion.
16.DisclaimerofWarranty.
THEREISNOWARRANTYFORTHEPROGRAM,TOTHEEXTENTPERMITTEDBYAPPLICABLE
LAW.EXCEPTWHENOTHERWISESTATEDINWRITINGTHECOPYRIGHTHOLDERSAND/OR
OTHERPARTIESPROVIDETHEPROGRAMASISWITHOUTWARRANTYOFANYKIND,EITHER
EXPRESSEDORIMPLIED,INCLUDING,BUTNOTLIMITEDTO,THEIMPLIEDWARRANTIESOF
MERCHANTABILITYANDFITNESSFORAPARTICULARPURPOSE.THEENTIRERISKASTOTHE
QUALITYANDPERFORMANCEOFTHEPROGRAMISWITHYOU.SHOULDTHEPROGRAMPROVE
DEFECTIVE,YOUASSUMETHECOSTOFALLNECESSARYSERVICING,REPAIRORCORRECTION.
17.LimitationofLiability.
INNOEVENTUNLESSREQUIREDBYAPPLICABLELAWORAGREEDTOINWRITINGWILLANY
COPYRIGHTHOLDER,ORANYOTHERPARTYWHOMODIFIESAND/ORCONVEYSTHEPROGRAM
ASPERMITTEDABOVE,BELIABLETOYOUFORDAMAGES,INCLUDINGANYGENERAL,
SPECIAL,INCIDENTALORCONSEQUENTIALDAMAGESARISINGOUTOFTHEUSEORINABILITY
TOUSETHEPROGRAM(INCLUDINGBUTNOTLIMITEDTOLOSSOFDATAORDATABEING
RENDEREDINACCURATEORLOSSESSUSTAINEDBYYOUORTHIRDPARTIESORAFAILUREOF
THEPROGRAMTOOPERATEWITHANYOTHERPROGRAMS),EVENIFSUCHHOLDEROROTHER
PARTYHASBEENADVISEDOFTHEPOSSIBILITYOFSUCHDAMAGES.
18.InterpretationofSections15and16.
Ifthedisclaimerofwarrantyandlimitationofliabilityprovidedabovecannotbegivenlocallegaleffect
accordingtotheirterms,reviewingcourtsshallapplylocallawthatmostcloselyapproximatesanabsolutewaiver
ofallcivilliabilityinconnectionwiththeProgram,unlessawarrantyorassumptionofliabilityaccompaniesa
copyofthePrograminreturnforafee.
ENDOFTERMSANDCONDITIONS
HowtoApplyTheseTermstoYourNewPrograms
Ifyoudevelopanewprogram,andyouwantittobeofthegreatestpossibleusetothepublic,thebestwaytoachieve
thisistomakeitfreesoftwarewhicheveryonecanredistributeandchangeundertheseterms.
Todoso,attachthefollowingnoticestotheprogram.Itissafesttoattachthemtothestartofeachsourcefiletomost
effectivelystatetheexclusionofwarrantyandeachfileshouldhaveatleastthecopyrightlineandapointertowhere
thefullnoticeisfound.
onelinetogivetheprogram'snameandabriefideaofwhatitdoes.
Copyright(C)yearnameofauthor
Thisprogramisfreesoftware:youcanredistributeitand/ormodify
itunderthetermsoftheGNUGeneralPublicLicenseaspublishedby
theFreeSoftwareFoundation,eitherversion3oftheLicense,or(at
youroption)anylaterversion.
Thisprogramisdistributedinthehopethatitwillbeuseful,but
WITHOUTANYWARRANTY;withouteventheimpliedwarrantyof
406/479
29/7/2016
TheGNUAwkUsersGuide
MERCHANTABILITYorFITNESSFORAPARTICULARPURPOSE.SeetheGNU
GeneralPublicLicenseformoredetails.
YoushouldhavereceivedacopyoftheGNUGeneralPublicLicense
alongwiththisprogram.Ifnot,seehttp://www.gnu.org/licenses/.
Alsoaddinformationonhowtocontactyoubyelectronicandpapermail.
Iftheprogramdoesterminalinteraction,makeitoutputashortnoticelikethiswhenitstartsinaninteractivemode:
programCopyright(C)yearnameofauthor
ThisprogramcomeswithABSOLUTELYNOWARRANTY;fordetailstypeshoww.
Thisisfreesoftware,andyouarewelcometoredistributeit
undercertainconditions;typeshowcfordetails.
ThehypotheticalcommandsshowwandshowcshouldshowtheappropriatepartsoftheGeneralPublicLicense.Of
course,yourprogramscommandsmightbedifferentforaGUIinterface,youwoulduseanaboutbox.
Youshouldalsogetyouremployer(ifyouworkasaprogrammer)orschool,ifany,tosignacopyrightdisclaimerfor
theprogram,ifnecessary.Formoreinformationonthis,andhowtoapplyandfollowtheGNUGPL,see
http://www.gnu.org/licenses/.
TheGNUGeneralPublicLicensedoesnotpermitincorporatingyourprogramintoproprietaryprograms.Ifyour
programisasubroutinelibrary,youmayconsideritmoreusefultopermitlinkingproprietaryapplicationswiththe
library.Ifthisiswhatyouwanttodo,usetheGNULesserGeneralPublicLicenseinsteadofthisLicense.Butfirst,
pleasereadhttp://www.gnu.org/philosophy/whynotlgpl.html.
Next:Index,Previous:Copying,Up:Top[Contents][Index]
Version1.3,3November2008
Copyright2000,2001,2002,2007,2008FreeSoftwareFoundation,Inc.
http://fsf.org/
Everyoneispermittedtocopyanddistributeverbatimcopies
ofthislicensedocument,butchangingitisnotallowed.
1.PREAMBLE
ThepurposeofthisLicenseistomakeamanual,textbook,orotherfunctionalandusefuldocumentfreeinthe
senseoffreedom:toassureeveryonetheeffectivefreedomtocopyandredistributeit,withorwithoutmodifying
it,eithercommerciallyornoncommercially.Secondarily,thisLicensepreservesfortheauthorandpublishera
waytogetcreditfortheirwork,whilenotbeingconsideredresponsibleformodificationsmadebyothers.
ThisLicenseisakindofcopyleft,whichmeansthatderivativeworksofthedocumentmustthemselvesbefree
inthesamesense.ItcomplementstheGNUGeneralPublicLicense,whichisacopyleftlicensedesignedforfree
software.
WehavedesignedthisLicenseinordertouseitformanualsforfreesoftware,becausefreesoftwareneedsfree
documentation:afreeprogramshouldcomewithmanualsprovidingthesamefreedomsthatthesoftwaredoes.
ButthisLicenseisnotlimitedtosoftwaremanualsitcanbeusedforanytextualwork,regardlessofsubject
matterorwhetheritispublishedasaprintedbook.WerecommendthisLicenseprincipallyforworkswhose
purposeisinstructionorreference.
2.APPLICABILITYANDDEFINITIONS
ThisLicenseappliestoanymanualorotherwork,inanymedium,thatcontainsanoticeplacedbythecopyright
holdersayingitcanbedistributedunderthetermsofthisLicense.Suchanoticegrantsaworldwide,royaltyfree
407/479
29/7/2016
TheGNUAwkUsersGuide
license,unlimitedinduration,tousethatworkundertheconditionsstatedherein.TheDocument,below,refers
toanysuchmanualorwork.Anymemberofthepublicisalicensee,andisaddressedasyou.Youacceptthe
licenseifyoucopy,modifyordistributetheworkinawayrequiringpermissionundercopyrightlaw.
AModifiedVersionoftheDocumentmeansanyworkcontainingtheDocumentoraportionofit,eithercopied
verbatim,orwithmodificationsand/ortranslatedintoanotherlanguage.
ASecondarySectionisanamedappendixorafrontmattersectionoftheDocumentthatdealsexclusivelywith
therelationshipofthepublishersorauthorsoftheDocumenttotheDocumentsoverallsubject(ortorelated
matters)andcontainsnothingthatcouldfalldirectlywithinthatoverallsubject.(Thus,iftheDocumentisinpart
atextbookofmathematics,aSecondarySectionmaynotexplainanymathematics.)Therelationshipcouldbea
matterofhistoricalconnectionwiththesubjectorwithrelatedmatters,oroflegal,commercial,philosophical,
ethicalorpoliticalpositionregardingthem.
TheInvariantSectionsarecertainSecondarySectionswhosetitlesaredesignated,asbeingthoseofInvariant
Sections,inthenoticethatsaysthattheDocumentisreleasedunderthisLicense.Ifasectiondoesnotfitthe
abovedefinitionofSecondarythenitisnotallowedtobedesignatedasInvariant.TheDocumentmaycontain
zeroInvariantSections.IftheDocumentdoesnotidentifyanyInvariantSectionsthentherearenone.
TheCoverTextsarecertainshortpassagesoftextthatarelisted,asFrontCoverTextsorBackCoverTexts,in
thenoticethatsaysthattheDocumentisreleasedunderthisLicense.AFrontCoverTextmaybeatmost5
words,andaBackCoverTextmaybeatmost25words.
ATransparentcopyoftheDocumentmeansamachinereadablecopy,representedinaformatwhose
specificationisavailabletothegeneralpublic,thatissuitableforrevisingthedocumentstraightforwardlywith
generictexteditorsor(forimagescomposedofpixels)genericpaintprogramsor(fordrawings)somewidely
availabledrawingeditor,andthatissuitableforinputtotextformattersorforautomatictranslationtoavarietyof
formatssuitableforinputtotextformatters.AcopymadeinanotherwiseTransparentfileformatwhosemarkup,
orabsenceofmarkup,hasbeenarrangedtothwartordiscouragesubsequentmodificationbyreadersisnot
Transparent.AnimageformatisnotTransparentifusedforanysubstantialamountoftext.Acopythatisnot
TransparentiscalledOpaque.
ExamplesofsuitableformatsforTransparentcopiesincludeplainASCIIwithoutmarkup,Texinfoinputformat,
LaTeXinputformat,SGMLorXMLusingapubliclyavailableDTD,andstandardconformingsimpleHTML,
PostScriptorPDFdesignedforhumanmodification.ExamplesoftransparentimageformatsincludePNG,XCF
andJPG.Opaqueformatsincludeproprietaryformatsthatcanbereadandeditedonlybyproprietaryword
processors,SGMLorXMLforwhichtheDTDand/orprocessingtoolsarenotgenerallyavailable,andthe
machinegeneratedHTML,PostScriptorPDFproducedbysomewordprocessorsforoutputpurposesonly.
TheTitlePagemeans,foraprintedbook,thetitlepageitself,plussuchfollowingpagesasareneededtohold,
legibly,thematerialthisLicenserequirestoappearinthetitlepage.Forworksinformatswhichdonothaveany
titlepageassuch,TitlePagemeansthetextnearthemostprominentappearanceoftheworkstitle,preceding
thebeginningofthebodyofthetext.
ThepublishermeansanypersonorentitythatdistributescopiesoftheDocumenttothepublic.
AsectionEntitledXYZmeansanamedsubunitoftheDocumentwhosetitleeitherispreciselyXYZor
containsXYZinparenthesesfollowingtextthattranslatesXYZinanotherlanguage.(HereXYZstandsfora
specificsectionnamementionedbelow,suchasAcknowledgements,Dedications,Endorsements,or
History.)ToPreservetheTitleofsuchasectionwhenyoumodifytheDocumentmeansthatitremainsa
sectionEntitledXYZaccordingtothisdefinition.
TheDocumentmayincludeWarrantyDisclaimersnexttothenoticewhichstatesthatthisLicenseappliestothe
Document.TheseWarrantyDisclaimersareconsideredtobeincludedbyreferenceinthisLicense,butonlyas
regardsdisclaimingwarranties:anyotherimplicationthattheseWarrantyDisclaimersmayhaveisvoidandhas
noeffectonthemeaningofthisLicense.
3.VERBATIMCOPYING
408/479
29/7/2016
TheGNUAwkUsersGuide
YoumaycopyanddistributetheDocumentinanymedium,eithercommerciallyornoncommercially,provided
thatthisLicense,thecopyrightnotices,andthelicensenoticesayingthisLicenseappliestotheDocumentare
reproducedinallcopies,andthatyouaddnootherconditionswhatsoevertothoseofthisLicense.Youmaynot
usetechnicalmeasurestoobstructorcontrolthereadingorfurthercopyingofthecopiesyoumakeordistribute.
However,youmayacceptcompensationinexchangeforcopies.Ifyoudistributealargeenoughnumberofcopies
youmustalsofollowtheconditionsinsection3.
Youmayalsolendcopies,underthesameconditionsstatedabove,andyoumaypubliclydisplaycopies.
4.COPYINGINQUANTITY
Ifyoupublishprintedcopies(orcopiesinmediathatcommonlyhaveprintedcovers)oftheDocument,
numberingmorethan100,andtheDocumentslicensenoticerequiresCoverTexts,youmustenclosethecopies
incoversthatcarry,clearlyandlegibly,alltheseCoverTexts:FrontCoverTextsonthefrontcover,andBack
CoverTextsonthebackcover.Bothcoversmustalsoclearlyandlegiblyidentifyyouasthepublisherofthese
copies.Thefrontcovermustpresentthefulltitlewithallwordsofthetitleequallyprominentandvisible.You
mayaddothermaterialonthecoversinaddition.Copyingwithchangeslimitedtothecovers,aslongasthey
preservethetitleoftheDocumentandsatisfytheseconditions,canbetreatedasverbatimcopyinginother
respects.
Iftherequiredtextsforeithercoveraretoovoluminoustofitlegibly,youshouldputthefirstoneslisted(asmany
asfitreasonably)ontheactualcover,andcontinuetherestontoadjacentpages.
IfyoupublishordistributeOpaquecopiesoftheDocumentnumberingmorethan100,youmusteitherincludea
machinereadableTransparentcopyalongwitheachOpaquecopy,orstateinorwitheachOpaquecopya
computernetworklocationfromwhichthegeneralnetworkusingpublichasaccesstodownloadusingpublic
standardnetworkprotocolsacompleteTransparentcopyoftheDocument,freeofaddedmaterial.Ifyouusethe
latteroption,youmusttakereasonablyprudentsteps,whenyoubegindistributionofOpaquecopiesinquantity,
toensurethatthisTransparentcopywillremainthusaccessibleatthestatedlocationuntilatleastoneyearafter
thelasttimeyoudistributeanOpaquecopy(directlyorthroughyouragentsorretailers)ofthateditiontothe
public.
Itisrequested,butnotrequired,thatyoucontacttheauthorsoftheDocumentwellbeforeredistributinganylarge
numberofcopies,togivethemachancetoprovideyouwithanupdatedversionoftheDocument.
5.MODIFICATIONS
YoumaycopyanddistributeaModifiedVersionoftheDocumentundertheconditionsofsections2and3above,
providedthatyoureleasetheModifiedVersionunderpreciselythisLicense,withtheModifiedVersionfillingthe
roleoftheDocument,thuslicensingdistributionandmodificationoftheModifiedVersiontowhoeverpossesses
acopyofit.Inaddition,youmustdothesethingsintheModifiedVersion:
1.UseintheTitlePage(andonthecovers,ifany)atitledistinctfromthatoftheDocument,andfromthose
ofpreviousversions(whichshould,iftherewereany,belistedintheHistorysectionoftheDocument).
Youmayusethesametitleasapreviousversioniftheoriginalpublisherofthatversiongivespermission.
2.ListontheTitlePage,asauthors,oneormorepersonsorentitiesresponsibleforauthorshipofthe
modificationsintheModifiedVersion,togetherwithatleastfiveoftheprincipalauthorsoftheDocument
(allofitsprincipalauthors,ifithasfewerthanfive),unlesstheyreleaseyoufromthisrequirement.
3.StateontheTitlepagethenameofthepublisheroftheModifiedVersion,asthepublisher.
4.PreserveallthecopyrightnoticesoftheDocument.
5.Addanappropriatecopyrightnoticeforyourmodificationsadjacenttotheothercopyrightnotices.
6.Include,immediatelyafterthecopyrightnotices,alicensenoticegivingthepublicpermissiontousethe
ModifiedVersionunderthetermsofthisLicense,intheformshownintheAddendumbelow.
7.PreserveinthatlicensenoticethefulllistsofInvariantSectionsandrequiredCoverTextsgiveninthe
Documentslicensenotice.
8.IncludeanunalteredcopyofthisLicense.
9.PreservethesectionEntitledHistory,PreserveitsTitle,andaddtoitanitemstatingatleastthetitle,year,
newauthors,andpublisheroftheModifiedVersionasgivenontheTitlePage.Ifthereisnosection
409/479
29/7/2016
TheGNUAwkUsersGuide
EntitledHistoryintheDocument,createonestatingthetitle,year,authors,andpublisherofthe
DocumentasgivenonitsTitlePage,thenaddanitemdescribingtheModifiedVersionasstatedinthe
previoussentence.
10.Preservethenetworklocation,ifany,givenintheDocumentforpublicaccesstoaTransparentcopyofthe
Document,andlikewisethenetworklocationsgivenintheDocumentforpreviousversionsitwasbased
on.ThesemaybeplacedintheHistorysection.Youmayomitanetworklocationforaworkthatwas
publishedatleastfouryearsbeforetheDocumentitself,oriftheoriginalpublisheroftheversionitrefers
togivespermission.
11.ForanysectionEntitledAcknowledgementsorDedications,PreservetheTitleofthesection,and
preserveinthesectionallthesubstanceandtoneofeachofthecontributoracknowledgementsand/or
dedicationsgiventherein.
12.PreservealltheInvariantSectionsoftheDocument,unalteredintheirtextandintheirtitles.Section
numbersortheequivalentarenotconsideredpartofthesectiontitles.
13.DeleteanysectionEntitledEndorsements.SuchasectionmaynotbeincludedintheModifiedVersion.
14.DonotretitleanyexistingsectiontobeEntitledEndorsementsortoconflictintitlewithanyInvariant
Section.
15.PreserveanyWarrantyDisclaimers.
IftheModifiedVersionincludesnewfrontmattersectionsorappendicesthatqualifyasSecondarySectionsand
containnomaterialcopiedfromtheDocument,youmayatyouroptiondesignatesomeorallofthesesectionsas
invariant.Todothis,addtheirtitlestothelistofInvariantSectionsintheModifiedVersionslicensenotice.
Thesetitlesmustbedistinctfromanyothersectiontitles.
YoumayaddasectionEntitledEndorsements,provideditcontainsnothingbutendorsementsofyourModified
Versionbyvariouspartiesforexample,statementsofpeerrevieworthatthetexthasbeenapprovedbyan
organizationastheauthoritativedefinitionofastandard.
YoumayaddapassageofuptofivewordsasaFrontCoverText,andapassageofupto25wordsasaBack
CoverText,totheendofthelistofCoverTextsintheModifiedVersion.OnlyonepassageofFrontCoverText
andoneofBackCoverTextmaybeaddedby(orthrougharrangementsmadeby)anyoneentity.IftheDocument
alreadyincludesacovertextforthesamecover,previouslyaddedbyyouorbyarrangementmadebythesame
entityyouareactingonbehalfof,youmaynotaddanotherbutyoumayreplacetheoldone,onexplicit
permissionfromthepreviouspublisherthataddedtheoldone.
Theauthor(s)andpublisher(s)oftheDocumentdonotbythisLicensegivepermissiontousetheirnamesfor
publicityforortoassertorimplyendorsementofanyModifiedVersion.
6.COMBININGDOCUMENTS
YoumaycombinetheDocumentwithotherdocumentsreleasedunderthisLicense,underthetermsdefinedin
section4aboveformodifiedversions,providedthatyouincludeinthecombinationalloftheInvariantSections
ofalloftheoriginaldocuments,unmodified,andlistthemallasInvariantSectionsofyourcombinedworkinits
licensenotice,andthatyoupreservealltheirWarrantyDisclaimers.
ThecombinedworkneedonlycontainonecopyofthisLicense,andmultipleidenticalInvariantSectionsmaybe
replacedwithasinglecopy.IftherearemultipleInvariantSectionswiththesamenamebutdifferentcontents,
makethetitleofeachsuchsectionuniquebyaddingattheendofit,inparentheses,thenameoftheoriginal
authororpublisherofthatsectionifknown,orelseauniquenumber.Makethesameadjustmenttothesection
titlesinthelistofInvariantSectionsinthelicensenoticeofthecombinedwork.
Inthecombination,youmustcombineanysectionsEntitledHistoryinthevariousoriginaldocuments,forming
onesectionEntitledHistorylikewisecombineanysectionsEntitledAcknowledgements,andanysections
EntitledDedications.YoumustdeleteallsectionsEntitledEndorsements.
7.COLLECTIONSOFDOCUMENTS
YoumaymakeacollectionconsistingoftheDocumentandotherdocumentsreleasedunderthisLicense,and
replacetheindividualcopiesofthisLicenseinthevariousdocumentswithasinglecopythatisincludedinthe
collection,providedthatyoufollowtherulesofthisLicenseforverbatimcopyingofeachofthedocumentsinall
410/479
29/7/2016
TheGNUAwkUsersGuide
otherrespects.
Youmayextractasingledocumentfromsuchacollection,anddistributeitindividuallyunderthisLicense,
providedyouinsertacopyofthisLicenseintotheextracteddocument,andfollowthisLicenseinallother
respectsregardingverbatimcopyingofthatdocument.
8.AGGREGATIONWITHINDEPENDENTWORKS
AcompilationoftheDocumentoritsderivativeswithotherseparateandindependentdocumentsorworks,inor
onavolumeofastorageordistributionmedium,iscalledanaggregateifthecopyrightresultingfromthe
compilationisnotusedtolimitthelegalrightsofthecompilationsusersbeyondwhattheindividualworks
permit.WhentheDocumentisincludedinanaggregate,thisLicensedoesnotapplytotheotherworksinthe
aggregatewhicharenotthemselvesderivativeworksoftheDocument.
IftheCoverTextrequirementofsection3isapplicabletothesecopiesoftheDocument,theniftheDocumentis
lessthanonehalfoftheentireaggregate,theDocumentsCoverTextsmaybeplacedoncoversthatbracketthe
Documentwithintheaggregate,ortheelectronicequivalentofcoversiftheDocumentisinelectronicform.
Otherwisetheymustappearonprintedcoversthatbracketthewholeaggregate.
9.TRANSLATION
Translationisconsideredakindofmodification,soyoumaydistributetranslationsoftheDocumentunderthe
termsofsection4.ReplacingInvariantSectionswithtranslationsrequiresspecialpermissionfromtheircopyright
holders,butyoumayincludetranslationsofsomeorallInvariantSectionsinadditiontotheoriginalversionsof
theseInvariantSections.YoumayincludeatranslationofthisLicense,andallthelicensenoticesinthe
Document,andanyWarrantyDisclaimers,providedthatyoualsoincludetheoriginalEnglishversionofthis
Licenseandtheoriginalversionsofthosenoticesanddisclaimers.Incaseofadisagreementbetweenthe
translationandtheoriginalversionofthisLicenseoranoticeordisclaimer,theoriginalversionwillprevail.
IfasectionintheDocumentisEntitledAcknowledgements,Dedications,orHistory,therequirement
(section4)toPreserveitsTitle(section1)willtypicallyrequirechangingtheactualtitle.
10.TERMINATION
Youmaynotcopy,modify,sublicense,ordistributetheDocumentexceptasexpresslyprovidedunderthis
License.Anyattemptotherwisetocopy,modify,sublicense,ordistributeitisvoid,andwillautomatically
terminateyourrightsunderthisLicense.
However,ifyouceaseallviolationofthisLicense,thenyourlicensefromaparticularcopyrightholderis
reinstated(a)provisionally,unlessanduntilthecopyrightholderexplicitlyandfinallyterminatesyourlicense,
and(b)permanently,ifthecopyrightholderfailstonotifyyouoftheviolationbysomereasonablemeanspriorto
60daysafterthecessation.
Moreover,yourlicensefromaparticularcopyrightholderisreinstatedpermanentlyifthecopyrightholder
notifiesyouoftheviolationbysomereasonablemeans,thisisthefirsttimeyouhavereceivednoticeofviolation
ofthisLicense(foranywork)fromthatcopyrightholder,andyoucuretheviolationpriorto30daysafteryour
receiptofthenotice.
Terminationofyourrightsunderthissectiondoesnotterminatethelicensesofpartieswhohavereceivedcopies
orrightsfromyouunderthisLicense.Ifyourrightshavebeenterminatedandnotpermanentlyreinstated,receipt
ofacopyofsomeorallofthesamematerialdoesnotgiveyouanyrightstouseit.
11.FUTUREREVISIONSOFTHISLICENSE
TheFreeSoftwareFoundationmaypublishnew,revisedversionsoftheGNUFreeDocumentationLicensefrom
timetotime.Suchnewversionswillbesimilarinspirittothepresentversion,butmaydifferindetailtoaddress
newproblemsorconcerns.Seehttp://www.gnu.org/copyleft/.
EachversionoftheLicenseisgivenadistinguishingversionnumber.IftheDocumentspecifiesthataparticular
411/479
29/7/2016
TheGNUAwkUsersGuide
numberedversionofthisLicenseoranylaterversionappliestoit,youhavetheoptionoffollowingtheterms
andconditionseitherofthatspecifiedversionorofanylaterversionthathasbeenpublished(notasadraft)by
theFreeSoftwareFoundation.IftheDocumentdoesnotspecifyaversionnumberofthisLicense,youmay
chooseanyversioneverpublished(notasadraft)bytheFreeSoftwareFoundation.IftheDocumentspecifies
thataproxycandecidewhichfutureversionsofthisLicensecanbeused,thatproxyspublicstatementof
acceptanceofaversionpermanentlyauthorizesyoutochoosethatversionfortheDocument.
12.RELICENSING
MassiveMultiauthorCollaborationSite(orMMCSite)meansanyWorldWideWebserverthatpublishes
copyrightableworksandalsoprovidesprominentfacilitiesforanybodytoeditthoseworks.Apublicwikithat
anybodycaneditisanexampleofsuchaserver.AMassiveMultiauthorCollaboration(orMMC)contained
inthesitemeansanysetofcopyrightableworksthuspublishedontheMMCsite.
CCBYSAmeanstheCreativeCommonsAttributionShareAlike3.0licensepublishedbyCreativeCommons
Corporation,anotforprofitcorporationwithaprincipalplaceofbusinessinSanFrancisco,California,aswellas
futurecopyleftversionsofthatlicensepublishedbythatsameorganization.
IncorporatemeanstopublishorrepublishaDocument,inwholeorinpart,aspartofanotherDocument.
AnMMCiseligibleforrelicensingifitislicensedunderthisLicense,andifallworksthatwerefirstpublished
underthisLicensesomewhereotherthanthisMMC,andsubsequentlyincorporatedinwholeorinpartintothe
MMC,(1)hadnocovertextsorinvariantsections,and(2)werethusincorporatedpriortoNovember1,2008.
TheoperatorofanMMCSitemayrepublishanMMCcontainedinthesiteunderCCBYSAonthesamesiteat
anytimebeforeAugust1,2009,providedtheMMCiseligibleforrelicensing.
ADDENDUM:HowtousethisLicenseforyourdocuments
TousethisLicenseinadocumentyouhavewritten,includeacopyoftheLicenseinthedocumentandputthe
followingcopyrightandlicensenoticesjustafterthetitlepage:
Copyright(C)yearyourname.
Permissionisgrantedtocopy,distributeand/ormodifythisdocument
underthetermsoftheGNUFreeDocumentationLicense,Version1.3
oranylaterversionpublishedbytheFreeSoftwareFoundation;
withnoInvariantSections,noFrontCoverTexts,andnoBackCover
Texts.Acopyofthelicenseisincludedinthesectionentitled``GNU
FreeDocumentationLicense''.
IfyouhaveInvariantSections,FrontCoverTextsandBackCoverTexts,replacethewithTexts.linewiththis:
withtheInvariantSectionsbeinglisttheirtitles,with
theFrontCoverTextsbeinglist,andwiththeBackCoverTexts
beinglist.
IfyouhaveInvariantSectionswithoutCoverTexts,orsomeothercombinationofthethree,mergethosetwo
alternativestosuitthesituation.
Ifyourdocumentcontainsnontrivialexamplesofprogramcode,werecommendreleasingtheseexamplesinparallel
underyourchoiceoffreesoftwarelicense,suchastheGNUGeneralPublicLicense,topermittheiruseinfree
software.
Previous:GNUFreeDocumentationLicense,Up:Top[Contents][Index]
Index
Jumpto: !"#$%&'(*+,./<=>?@[\^_{|~
ABCDEFGHIJKLMNOPQRSTUVWXYZ
412/479
29/7/2016
TheGNUAwkUsersGuide
IndexEntry
Section
!(exclamationpoint),!operator:
BooleanOps
Precedence
Ranges
EgrepProgram
ComparisonOperators
Precedence
RegexpUsage
ComputedRegexps
Casesensitivity
RegexpConstants
ComparisonOperators
ComparisonOperators
Precedence
ExpressionPatterns
!
!(exclamationpoint),!=operator:
!(exclamationpoint),!=operator:
!(exclamationpoint),!~operator:
"
"(doublequote),inregexpconstants:
"(doublequote),inshellcommands:
ComputedRegexps
Quoting
#
#(numbersign),#!(executablescripts):
#(numbersign),commenting:
ExecutableScripts
Comments
$
$(dollarsign),$fieldoperator:
$(dollarsign),$fieldoperator:
$(dollarsign),incrementingfieldsandarrays:
$(dollarsign),regexpoperator:
Fields
Precedence
IncrementOps
RegexpOperators
%
%(percentsign),%operator:
%(percentsign),%=operator:
%(percentsign),%=operator:
Precedence
AssignmentOps
Precedence
&
&(ampersand),&&operator:
&(ampersand),&&operator:
&(ampersand),gsub()/gensub()/sub()functionsand:
BooleanOps
Precedence
GoryDetails
'
'(singlequote):
'(singlequote)ingawkcommandlines:
'(singlequote),inshellcommands:
'(singlequote),vs.apostrophe:
Oneshot
Long
Quoting
Comments
413/479
29/7/2016
TheGNUAwkUsersGuide
'(singlequote),withdoublequotes:
Quoting
()(parentheses),inaprofile:
Profiling
RegexpOperators
(
()(parentheses),regexpoperator:
*
*(asterisk),*operator,asmultiplicationoperator:
*(asterisk),*operator,asregexpoperator:
*(asterisk),*operator,nullstrings,matching:
*(asterisk),**operator:
*(asterisk),**operator:
*(asterisk),**=operator:
*(asterisk),**=operator:
*(asterisk),*=operator:
*(asterisk),*=operator:
Precedence
RegexpOperators
StringFunctions
ArithmeticOps
Precedence
AssignmentOps
Precedence
AssignmentOps
Precedence
+
+(plussign),+operator:
+(plussign),regexpoperator:
Precedence
Precedence
IncrementOps
IncrementOps
Precedence
AssignmentOps
Precedence
RegexpOperators
,(comma),inrangepatterns:
Ranges
(hyphen),operator:
Precedence
Precedence
IncrementOps
Precedence
AssignmentOps
Precedence
Options
BracketExpressions
Options
Options
Options
Options
Options
AdditionalConfigurationOptions
+(plussign),+operator:
+(plussign),++operator:
+(plussign),+=operator:
+(plussign),+=operator:
(hyphen),operator:
(hyphen),operator:
(hyphen),operator:
(hyphen),=operator:
(hyphen),=operator:
(hyphen),filenamesbeginningwith:
(hyphen),inbracketexpressions:
assignoption:
bignumoption:
charactersasbytesoption:
copyrightoption:
debugoption:
disableextensionsconfigurationoption:
disablelintconfigurationoption:
disablenlsconfigurationoption:
414/479
29/7/2016
TheGNUAwkUsersGuide
dumpvariablesoption:
Options
dumpvariablesoption,usingforlibraryfunctions:
LibraryNames
Options
Options
Options
Options
StringExtraction
StringExtraction
Options
Options
CommandLine
Options
Options
Options
Options
NondecimalData
NondecimalData
Options
Options
Options
Options
Options
Profiling
Options
Options
I/OFunctions
Getline
Redirection
Options
Options
Options
Options
Options
Options
Options
Options
Options
Options
Options
Options
Options
Long
Options
Options
Options
execoption:
fieldseparatoroption:
fileoption:
genpotoption:
genpotoption:
genpotoption:
helpoption:
includeoption:
lintoption:
lintoption:
lintoldoption:
loadoption:
nondecimaldataoption:
nondecimaldataoption:
nondecimaldataoption,strtonum()functionand:
optimizeoption:
posixoption:
posixoption,traditionaloptionand:
prettyprintoption:
profileoption:
profileoption:
reintervaloption:
sandboxoption:
sandboxoption,disablingsystem()function:
sandboxoption,inputredirectionwithgetline:
sandboxoption,outputredirectionwithprint,printf:
sourceoption:
traditionaloption:
traditionaloption,posixoptionand:
uselcnumericoption:
versionoption:
withwhinyuserstrftimeconfigurationoption:
boption:
coption:
Coption:
doption:
Doption:
eoption:
Eoption:
eoption:
foption:
Foption:
foption:
Foption,FtsetsFStoTAB:
415/479
29/7/2016
TheGNUAwkUsersGuide
Foption,commandline:
foption,multipleuses:
goption:
hoption:
ioption:
loption:
loption:
Loption:
Moption:
noption:
Noption:
ooption:
Ooption:
poption:
Poption:
roption:
Soption:
voption:
Voption:
voption:
Woption:
CommandLineFieldSeparator
Options
Options
Options
Options
Options
Options
Options
Options
Options
Options
Options
Options
Options
Options
Options
Options
Options
Options
AssignmentOptions
Options
.
.(period),regexpoperator:
.gmofiles:
.gmofiles,specifyingdirectoryof:
.gmofiles,specifyingdirectoryof:
.mofiles,convertingfrom.po:
.pofiles:
.pofiles:
.pofiles,convertingto.mo:
.potfiles:
RegexpOperators
Explaininggettext
Explaininggettext
Programmeri18n
I18NExample
Explaininggettext
Translatori18n
I18NExample
Explaininggettext
/
/(forwardslash)toencloseregularexpressions:
/(forwardslash),/operator:
/(forwardslash),/=operator:
/(forwardslash),/=operator:
/(forwardslash),/=operator,vs./=/regexpconstant:
/(forwardslash),patternsand:
/=operatorvs./=/regexpconstant:
/dev/specialfiles:
/dev/fd/Nspecialfiles(gawk):
/inet/specialfiles(gawk):
/inet4/specialfiles(gawk):
/inet6/specialfiles(gawk):
Regexp
Precedence
AssignmentOps
Precedence
AssignmentOps
ExpressionPatterns
AssignmentOps
SpecialFD
SpecialFD
TCP/IPNetworking
TCP/IPNetworking
TCP/IPNetworking
416/479
29/7/2016
TheGNUAwkUsersGuide
;(semicolon),AWKPATHvariableand:
PCUsing
;(semicolon),separatingstatementsinactions:
Statements/Lines
ActionOverview
Statements
<
<(leftanglebracket),<operator:
<(leftanglebracket),<operator:
<(leftanglebracket),<operator(I/O):
<(leftanglebracket),<=operator:
<(leftanglebracket),<=operator:
ComparisonOperators
Precedence
Getline/File
ComparisonOperators
Precedence
=
=(equalssign),=operator:
=(equalssign),==operator:
=(equalssign),==operator:
AssignmentOps
ComparisonOperators
Precedence
>
>(rightanglebracket),>operator:
>(rightanglebracket),>operator:
>(rightanglebracket),>operator(I/O):
>(rightanglebracket),>=operator:
>(rightanglebracket),>=operator:
>(rightanglebracket),>>operator(I/O):
>(rightanglebracket),>>operator(I/O):
ComparisonOperators
Precedence
Redirection
ComparisonOperators
Precedence
Redirection
Precedence
?
?(questionmark),?:operator:
?(questionmark),regexpoperator:
?(questionmark),regexpoperator:
Precedence
RegexpOperators
GNURegexpOperators
@
@notationforindirectfunctioncalls:
@loaddirective:
IndirectCalls
IncludeFiles
LoadingSharedLibraries
[](squarebrackets),regexpoperator:
RegexpOperators
\(backslash):
Comments
Statements/Lines
Statements/Lines
Statements/Lines
@includedirective:
\
\(backslash),asfieldseparator:
\(backslash),continuinglinesand:
\(backslash),continuinglinesand,commentsand:
\(backslash),continuinglinesand,incsh:
417/479
29/7/2016
TheGNUAwkUsersGuide
\(backslash),gsub()/gensub()/sub()functionsand:
\(backslash),inbracketexpressions:
\(backslash),inescapesequences:
\(backslash),inescapesequences:
\(backslash),inescapesequences,POSIXand:
\(backslash),inregexpconstants:
\(backslash),inshellcommands:
\(backslash),regexpoperator:
\(backslash),\"escapesequence:
\(backslash),\'operator(gawk):
\(backslash),\/escapesequence:
\(backslash),\<operator(gawk):
\(backslash),\>operator(gawk):
\(backslash),\aescapesequence:
\(backslash),\bescapesequence:
\(backslash),\Boperator(gawk):
\(backslash),\fescapesequence:
\(backslash),\nescapesequence:
\(backslash),\nnnescapesequence:
\(backslash),\rescapesequence:
\(backslash),\soperator(gawk):
\(backslash),\Soperator(gawk):
\(backslash),\tescapesequence:
\(backslash),\vescapesequence:
\(backslash),\woperator(gawk):
\(backslash),\Woperator(gawk):
\(backslash),\xescapesequence:
\(backslash),\yoperator(gawk):
\(backslash),\òperator(gawk):
GoryDetails
BracketExpressions
EscapeSequences
EscapeSequences
EscapeSequences
ComputedRegexps
Quoting
RegexpOperators
EscapeSequences
GNURegexpOperators
EscapeSequences
GNURegexpOperators
GNURegexpOperators
EscapeSequences
EscapeSequences
GNURegexpOperators
EscapeSequences
EscapeSequences
EscapeSequences
EscapeSequences
GNURegexpOperators
GNURegexpOperators
EscapeSequences
EscapeSequences
GNURegexpOperators
GNURegexpOperators
EscapeSequences
GNURegexpOperators
GNURegexpOperators
^
^(caret),inbracketexpressions:
^(caret),inFS:
^(caret),regexpoperator:
^(caret),regexpoperator:
^(caret),ôperator:
^(caret),^=operator:
^(caret),^=operator:
BracketExpressions
RegexpFieldSplitting
RegexpOperators
GNURegexpOperators
Precedence
AssignmentOps
Precedence
_
_(underscore),Cmacro:
_(underscore),innamesofprivatevariables:
_(underscore),translatablestring:
_gr_init()userdefinedfunction:
_ord_init()userdefinedfunction:
_pw_init()userdefinedfunction:
Explaininggettext
LibraryNames
Programmeri18n
GroupFunctions
OrdinalFunctions
PasswdFunctions
418/479
29/7/2016
TheGNUAwkUsersGuide
{
{}(braces):
{}(braces),actionsand:
{}(braces),statements,grouping:
Profiling
ActionOverview
Statements
|
|(verticalbar):
|(verticalbar),|operator(I/O):
|(verticalbar),|&operator(I/O):
|(verticalbar),|&operator(I/O),pipes,closing:
|(verticalbar),||operator:
|(verticalbar),||operator:
RegexpOperators
Getline/Pipe
Redirection
Precedence
Getline/Coprocess
Redirection
Precedence
TwowayI/O
CloseFilesAndPipes
BooleanOps
Precedence
~
~(tilde),~operator:
~(tilde),~operator:
RegexpUsage
ComputedRegexps
Casesensitivity
RegexpConstants
ComparisonOperators
ComparisonOperators
Precedence
ExpressionPatterns
accessingfields:
accessingglobalvariablesfromextensions:
accountinformation:
accountinformation:
actions:
actions,controlstatementsin:
actions,default:
actions,empty:
Adaprogramminglanguage:
adding,featurestogawk:
adding,fields:
advancedfeatures,fixedwidthdata:
advancedfeatures,gawk:
advancedfeatures,networkprogramming:
advancedfeatures,nondecimalinputdata:
advancedfeatures,processes,communicatingwith:
advancedfeatures,specifyingfieldcontent:
Aho,Alfred:
Fields
SymbolTableAccess
PasswdFunctions
GroupFunctions
ActionOverview
Statements
VerySimple
VerySimple
Glossary
AddingCode
ChangingFields
ConstantSize
AdvancedFeatures
TCP/IPNetworking
NondecimalData
TwowayI/O
SplittingByContent
History
~(tilde),~operator:
~(tilde),~operator:
~(tilde),~operator:
~(tilde),~operator:
~(tilde),~operator:
~(tilde),~operator:
419/479
29/7/2016
TheGNUAwkUsersGuide
Aho,Alfred:
alarmclockexampleprogram:
Contributors
AlarmProgram
alarm.awkprogram:
AlarmProgram
BasicHighLevel
MemoryAllocationFunctions
Glossary
Glossary
AssignmentOps
BooleanOps
Precedence
GoryDetails
AnagramProgram
AnagramProgram
BitwiseFunctions
BooleanOps
BitwiseFunctions
Glossary
ExtensionAPIInformationalVariables
ExtensionVersioning
ArbitraryPrecisionArithmetic
ArbitraryPrecisionIntegers
Bugs
NumericFunctions
Autoset
OtherArguments
ARGCandARGV
ExecutableScripts
Autoset
OtherArguments
OtherArguments
Autoset
ARGCandARGV
CommandLine
FunctionCalls
GetoptFunction
OtherArguments
ArithmeticOps
ArrayManipulation
ReferencetoElements
ControllingScanning
StringFunctions
Arrays
ArraysofArrays
ArrayExample
ArrayIntro
PassByValue/Reference
ArrayIntro
algorithms:
allocatingmemoryforextensions:
amazingawkassembler(aaa):
amazinglyworkableformatter(awf):
ambiguity,syntactic:/=operatorvs./=/regexpconstant:
ampersand(&),&&operator:
ampersand(&),&&operator:
ampersand(&),gsub()/gensub()/sub()functionsand:
anagram.awkprogram:
anagrams,finding:
ANDbitwiseoperation:
andBooleanlogicoperator:
and()function(gawk):
ANSI:
APIinformationalvariables:
APIversion:
arbitraryprecision:
arbitraryprecisionintegers:
archaeologists:
arctangent:
ARGC/ARGVvariables:
ARGC/ARGVvariables,commandlinearguments:
ARGC/ARGVvariables,howtouse:
ARGC/ARGVvariables,portabilityand:
ARGINDvariable:
ARGINDvariable,commandlinearguments:
arguments,commandline:
arguments,commandline,invokingawk:
arguments,infunctioncalls:
arguments,processing:
ARGVarray,indexinginto:
arithmeticoperators:
arraymanipulationinextensions:
arraymembers:
arrayscanningorder,controlling:
array,numberofelements:
arrays:
arraysofarrays:
arrays,anexampleofusing:
arrays,andIGNORECASEvariable:
arrays,asparameterstofunctions:
arrays,associative:
420/479
29/7/2016
arrays,associative,libraryfunctionsand:
arrays,deletingentirecontents:
arrays,elementsthatdontexist:
arrays,elements,assigningvalues:
arrays,elements,deleting:
arrays,elements,orderofaccessbyinoperator:
arrays,elements,retrievingnumberof:
arrays,forstatementand:
arrays,indexing:
arrays,mergingintostrings:
arrays,multidimensional:
arrays,multidimensional,scanning:
arrays,numericsubscripts:
arrays,referencingelements:
arrays,scanning:
arrays,sorting:
arrays,sorting,andIGNORECASEvariable:
arrays,sparse:
arrays,subscripts,uninitializedvariablesas:
arrays,unassignedelements:
artificialintelligence,gawkand:
ASCII:
ASCII:
asort()function(gawk):
asort()function(gawk):
asort()function(gawk),arrays,sorting:
asorti()function(gawk):
asorti()function(gawk):
asorti()function(gawk),arrays,sorting:
assert()function(Clibrary):
assert()userdefinedfunction:
assertions:
assignvaluestovariables,indebugger:
assignmentoperators:
assignmentoperators,evaluationorder:
assignmentoperators,lvalues/rvalues:
assignmentsasfilenames:
associativearrays:
asterisk(*),*operator,asmultiplicationoperator:
asterisk(*),*operator,asregexpoperator:
asterisk(*),*operator,nullstrings,matching:
asterisk(*),**operator:
asterisk(*),**operator:
asterisk(*),**=operator:
asterisk(*),**=operator:
asterisk(*),*=operator:
asterisk(*),*=operator:
TheGNUAwkUsersGuide
LibraryNames
Delete
ReferencetoElements
AssigningElements
Delete
ScanninganArray
StringFunctions
ScanninganArray
ArrayIntro
JoinFunction
Multidimensional
Multiscanning
NumericArraySubscripts
ReferencetoElements
ScanninganArray
ArraySortingFunctions
ArrayIntro
UninitializedSubscripts
ReferencetoElements
Distributioncontents
OrdinalFunctions
Glossary
StringFunctions
StringFunctions
AssertFunction
AssertFunction
AssertFunction
ViewingAndChangingData
AssignmentOps
AssignmentOps
AssignmentOps
IgnoringAssigns
ArrayIntro
Precedence
RegexpOperators
StringFunctions
ArithmeticOps
Precedence
AssignmentOps
Precedence
AssignmentOps
Precedence
421/479
29/7/2016
atan2()function:
automaticdisplays,indebugger:
awf(amazinglyworkableformatter)program:
awkdebugging,enabling:
awklanguage,POSIXversion:
awkprofiling,enabling:
awkprograms:
awkprograms:
awkprograms:
awkprograms,complex:
awkprograms,documenting:
awkprograms,documenting:
awkprograms,examplesof:
awkprograms,executionof:
awkprograms,internationalizing:
awkprograms,internationalizing:
awkprograms,lengthy:
awkprograms,lengthy,assertions:
awkprograms,locationof:
awkprograms,onelineexamples:
awkprograms,profiling:
awkprograms,running:
awkprograms,running:
awkprograms,running,fromshellscripts:
awkprograms,running,withoutinputfiles:
awkprograms,shellvariablesin:
awk,functionof:
awk,gawkand:
awk,gawkand:
awk,historyof:
awk,implementationissues,pipes:
awk,implementations:
awk,implementations,limits:
awk,invoking:
awk,newvs.old:
awk,newvs.old,OFMTvariable:
awk,POSIXand:
awk,POSIXand,SeeAlsoPOSIXawk:
awk,regexpconstantsand:
awk,SeeAlsogawk:
awk,termsdescribing:
awk,usesfor:
awk,usesfor:
awk,usesfor:
awk,versionsof:
TheGNUAwkUsersGuide
NumericFunctions
DebuggerInfo
Glossary
Options
AssignmentOps
Options
GettingStarted
ExecutableScripts
TwoRules
When
Comments
LibraryNames
SamplePrograms
NextStatement
I18NFunctions
Programmeri18n
Long
AssertFunction
Options
Options
Options
VerySimple
Profiling
Runninggawk
Long
Oneshot
ReadTerminal
UsingShellVariables
GettingStarted
Preface
ThisManual
History
Redirection
OtherVersions
GetlineNotes
CommandLine
Names
StringsAndNumbers
Preface
Preface
ComparisonOperators
Preface
ThisManual
Preface
GettingStarted
When
V7/SVR3.1
422/479
29/7/2016
TheGNUAwkUsersGuide
awk,versionsof,changesbetweenSVR3.1andSVR4:
awk,versionsof,changesbetweenSVR4andPOSIXawk:
awk,versionsof,changesbetweenV7andSVR3.1:
awk,versionsof,SeeAlsoBrianKernighansawk:
awk,versionsof,SeeAlsoBrianKernighansawk:
awkacompilerforawk:
AWKLIBPATHenvironmentvariable:
AWKPATHenvironmentvariable:
AWKPATHenvironmentvariable:
awkprof.outfile:
awksed.awkprogram:
awkvars.outfile:
SVR4
POSIX
V7/SVR3.1
BTL
OtherVersions
OtherVersions
AWKLIBPATHVariable
AWKPATHVariable
PCUsing
Profiling
SimpleSed
Options
B
bdebuggercommand(aliasforbreak):
backslash(\):
backslash(\),asfieldseparator:
backslash(\),continuinglinesand:
backslash(\),continuinglinesand,commentsand:
backslash(\),continuinglinesand,incsh:
backslash(\),gsub()/gensub()/sub()functionsand:
backslash(\),inbracketexpressions:
backslash(\),inescapesequences:
backslash(\),inescapesequences:
backslash(\),inescapesequences,POSIXand:
backslash(\),inregexpconstants:
backslash(\),inshellcommands:
backslash(\),regexpoperator:
backslash(\),\"escapesequence:
backslash(\),\'operator(gawk):
backslash(\),\/escapesequence:
backslash(\),\<operator(gawk):
backslash(\),\>operator(gawk):
backslash(\),\aescapesequence:
backslash(\),\bescapesequence:
backslash(\),\Boperator(gawk):
backslash(\),\fescapesequence:
backslash(\),\nescapesequence:
backslash(\),\nnnescapesequence:
backslash(\),\rescapesequence:
backslash(\),\soperator(gawk):
backslash(\),\Soperator(gawk):
backslash(\),\tescapesequence:
backslash(\),\vescapesequence:
backslash(\),\woperator(gawk):
backslash(\),\Woperator(gawk):
backslash(\),\xescapesequence:
BreakpointControl
Comments
Statements/Lines
Statements/Lines
Statements/Lines
GoryDetails
BracketExpressions
EscapeSequences
EscapeSequences
EscapeSequences
ComputedRegexps
Quoting
RegexpOperators
EscapeSequences
GNURegexpOperators
EscapeSequences
GNURegexpOperators
GNURegexpOperators
EscapeSequences
EscapeSequences
GNURegexpOperators
EscapeSequences
EscapeSequences
EscapeSequences
EscapeSequences
GNURegexpOperators
GNURegexpOperators
EscapeSequences
EscapeSequences
GNURegexpOperators
GNURegexpOperators
EscapeSequences
423/479
29/7/2016
TheGNUAwkUsersGuide
backslash(\),\yoperator(gawk):
backslash(\),\òperator(gawk):
backtracedebuggercommand:
Beebe,NelsonH.F.:
Beebe,NelsonH.F.:
BEGINpattern:
BEGINpattern:
BEGINpattern:
BEGINpattern,andprofiling:
BEGINpattern,assert()userdefinedfunctionand:
BEGINpattern,Booleanpatternsand:
BEGINpattern,exitstatementand:
BEGINpattern,getlineand:
BEGINpattern,headings,adding:
BEGINpattern,next/nextfilestatementsand:
BEGINpattern,next/nextfilestatementsand:
BEGINpattern,OFS/ORSvariables,assigningvaluesto:
BEGINpattern,operatorsand:
BEGINpattern,printstatementand:
BEGINpattern,pwcatprogram:
BEGINpattern,runningawkprogramsand:
BEGINpattern,TEXTDOMAINvariableand:
BEGINFILEpattern:
BEGINFILEpattern,Booleanpatternsand:
beginfile()userdefinedfunction:
Bentley,Jon:
Benzinger,Michael:
Berry,Karl:
Berry,Karl:
Berry,Karl:
binaryinput/output:
bindtextdomain()function(Clibrary):
bindtextdomain()function(gawk):
bindtextdomain()function(gawk):
bindtextdomain()function(gawk),portabilityand:
BINMODEvariable:
BINMODEvariable:
bitmanipulationfunctions:
bits2str()userdefinedfunction:
bitwiseAND:
bitwisecomplement:
bitwiseOR:
bitwiseXOR:
bitwise,complement:
bitwise,operations:
bitwise,shift:
body,inactions:
GNURegexpOperators
GNURegexpOperators
ExecutionStack
Acknowledgments
OtherVersions
FieldSeparators
BEGIN/END
UsingBEGIN/END
Profiling
AssertFunction
ExpressionPatterns
ExitStatement
GetlineNotes
PrintExamples
I/OAndBEGIN/END
NextStatement
OutputSeparators
UsingBEGIN/END
I/OAndBEGIN/END
PasswdFunctions
CutProgram
Programmeri18n
BEGINFILE/ENDFILE
ExpressionPatterns
FiletransFunction
Glossary
Contributors
Acknowledgments
Acknowledgments
RangesandLocales
Usermodified
Explaininggettext
I18NFunctions
Programmeri18n
I18NPortability
Usermodified
PCUsing
BitwiseFunctions
BitwiseFunctions
BitwiseFunctions
BitwiseFunctions
BitwiseFunctions
BitwiseFunctions
BitwiseFunctions
BitwiseFunctions
BitwiseFunctions
Statements
424/479
29/7/2016
body,inloops:
Booleanexpressions:
Booleanexpressions,aspatterns:
Booleanoperators,SeeBooleanexpressions:
Bourneshell,quotingrulesfor:
braces({}):
braces({}),actionsand:
braces({}),statements,grouping:
bracketexpressions:
bracketexpressions:
bracketexpressions,characterclasses:
bracketexpressions,collatingelements:
bracketexpressions,collatingsymbols:
bracketexpressions,complemented:
bracketexpressions,equivalenceclasses:
bracketexpressions,nonASCII:
bracketexpressions,rangeexpressions:
breakdebuggercommand:
breakstatement:
breakpoint:
breakpointatlocation,howtodelete:
breakpointcommands:
breakpointcondition:
breakpoint,deletebynumber:
breakpoint,howtodisableorenable:
breakpoint,setting:
Brennan,Michael:
Brennan,Michael:
Brennan,Michael:
Brennan,Michael:
Brennan,Michael:
Brennan,Michael:
Brennan,Michael:
BrianKernighansawk:
BrianKernighansawk:
BrianKernighansawk:
BrianKernighansawk:
BrianKernighansawk:
BrianKernighansawk:
BrianKernighansawk:
BrianKernighansawk:
BrianKernighansawk:
BrianKernighansawk:
BrianKernighansawk:
BrianKernighansawk:
BrianKernighansawk:
TheGNUAwkUsersGuide
WhileStatement
BooleanOps
ExpressionPatterns
BooleanOps
Quoting
Profiling
ActionOverview
Statements
RegexpOperators
BracketExpressions
BracketExpressions
BracketExpressions
BracketExpressions
RegexpOperators
BracketExpressions
BracketExpressions
BracketExpressions
BreakpointControl
BreakStatement
DebuggingTerms
BreakpointControl
DebuggerExecutionControl
BreakpointControl
BreakpointControl
BreakpointControl
BreakpointControl
Foreword3
Foreword4
Acknowledgments
Delete
SimpleSed
OtherVersions
OtherVersions
When
EscapeSequences
GNURegexpOperators
Getline/Pipe
Concatenation
I/OAndBEGIN/END
BreakStatement
ContinueStatement
NextfileStatement
Delete
StringFunctions
GoryDetails
425/479
29/7/2016
TheGNUAwkUsersGuide
BrianKernighansawk:
BrianKernighansawk,extensions:
BrianKernighansawk,sourcecode:
Brini,Davide:
Brink,Jeroen:
Broder,AlanJ.:
Brown,Martin:
BSDbasedoperatingsystems:
btdebuggercommand(aliasforbacktrace):
Buening,Andreas:
Buening,Andreas:
Buening,Andreas:
buffering,input/output:
buffering,input/output:
buffering,interactivevs.noninteractive:
buffers,flushing:
buffers,flushing:
buffers,operatorsfor:
bugreports,emailaddress,buggawk@gnu.org:
buggawk@gnu.orgbugreportingaddress:
builtinfunctions:
builtinfunctions,evaluationorder:
BusyBoxAwk:
I/OFunctions
BTL
OtherVersions
SignatureProgram
DOSQuoting
Contributors
Contributors
Glossary
ExecutionStack
Acknowledgments
Contributors
Bugs
I/OFunctions
TwowayI/O
I/OFunctions
I/OFunctions
I/OFunctions
GNURegexpOperators
Bugs
Bugs
Functions
CallingBuiltin
OtherVersions
c.e.,Seecommonextensions:
callbyreference:
callbyvalue:
callstack,displayindebugger:
caret(^),inbracketexpressions:
caret(^),regexpoperator:
caret(^),regexpoperator:
caret(^),ôperator:
caret(^),^=operator:
caret(^),^=operator:
casekeyword:
casesensitivity,andregexps:
casesensitivity,andstringcomparisons:
casesensitivity,arrayindicesand:
casesensitivity,convertingcase:
casesensitivity,exampleprograms:
casesensitivity,gawk:
casesensitivity,regexpsand:
CGI,awkscriptsfor:
characterclasses,Seebracketexpressions:
characterlistsinregularexpression:
characterlists,Seebracketexpressions:
Conventions
ExecutionStack
BracketExpressions
RegexpOperators
GNURegexpOperators
Precedence
AssignmentOps
Precedence
SwitchStatement
Usermodified
Usermodified
ArrayIntro
StringFunctions
LibraryFunctions
Casesensitivity
Casesensitivity
Options
RegexpOperators
BracketExpressions
RegexpOperators
426/479
29/7/2016
charactersets(machinecharacterencodings):
charactersets(machinecharacterencodings):
charactersets,SeeAlsobracketexpressions:
characters,counting:
characters,transliterating:
characters,valuesofasnumbers:
Chassell,RobertJ.:
chdir()extensionfunction:
chemutility:
chr()extensionfunction:
chr()userdefinedfunction:
cleardebuggercommand:
Cliffrandomnumbers:
cliff_rand()userdefinedfunction:
closefileorcoprocess:
close()function:
close()function:
close()function,portability:
close()function,returnvalue:
close()function,twowaypipesand:
Close,Diane:
Close,Diane:
Collado,Manuel:
collatingelements:
collatingsymbols:
Colombo,Antonio:
Colombo,Antonio:
columns,aligning:
columns,cutting:
comma(,),inrangepatterns:
commandcompletion,indebugger:
commandline,arguments:
commandline,directorieson:
commandline,formats:
commandline,FSon,setting:
commandline,invokingawkfrom:
commandline,optionf:
commandline,options:
commandline,options,endof:
commandline,variables,assigningon:
commandlineoptions,processing:
commandlineoptions,stringextraction:
commandsdebuggercommand:
commandstoexecuteatbreakpoint:
TheGNUAwkUsersGuide
OrdinalFunctions
Glossary
RegexpOperators
WcProgram
TranslateProgram
OrdinalFunctions
Acknowledgments
ExtensionSampleFileFunctions
Glossary
ExtensionSampleOrd
OrdinalFunctions
BreakpointControl
CliffRandomFunction
CliffRandomFunction
I/OFunctions
CloseFilesAndPipes
I/OFunctions
CloseFilesAndPipes
CloseFilesAndPipes
TwowayI/O
ManualHistory
Contributors
Acknowledgments
BracketExpressions
BracketExpressions
Acknowledgments
Contributors
PrintExamples
CutProgram
Ranges
ReadlineSupport
OtherArguments
Autoset
ARGCandARGV
Commandlinedirectories
Runninggawk
CommandLine
Long
Options
Options
AssignmentOptions
GetoptFunction
StringExtraction
427/479
29/7/2016
TheGNUAwkUsersGuide
commenting:
commenting,backslashcontinuationand:
commonextensions,**operator:
commonextensions,**=operator:
commonextensions,/dev/stderrspecialfile:
commonextensions,/dev/stdinspecialfile:
commonextensions,/dev/stdoutspecialfile:
commonextensions,BINMODEvariable:
commonextensions,deletetodeleteentirearrays:
commonextensions,funckeyword:
commonextensions,length()appliedtoanarray:
commonextensions,RSasaregexp:
commonextensions,singlecharacterfields:
commonextensions,\xescapesequence:
comp.lang.awknewsgroup:
comparisonexpressions:
comparisonexpressions,aspatterns:
comparisonexpressions,stringvs.regexp:
compatibilitymode(gawk),extensions:
compatibilitymode(gawk),filenames:
compatibilitymode(gawk),hexadecimalnumbers:
compatibilitymode(gawk),octalnumbers:
compatibilitymode(gawk),specifying:
compiledprograms:
compiledprograms:
compilinggawkforCygwin:
compilinggawkforMSDOSandMSWindows:
compilinggawkforVMS:
compilinggawkwithEMXforOS/2:
compl()function(gawk):
complement,bitwise:
compoundstatements,controlstatementsand:
concatenating:
conditiondebuggercommand:
conditionalexpressions:
configurationoption,disableextensions:
configurationoption,disablelint:
configurationoption,disablenls:
configurationoption,withwhinyuserstrftime:
configurationoptions,gawk:
constantregexps:
constants,nondecimal:
constants,numeric:
constants,typesof:
continueprogram,indebugger:
continuestatement:
Comments
Statements/Lines
ArithmeticOps
AssignmentOps
SpecialFD
SpecialFD
SpecialFD
PCUsing
Delete
DefinitionSyntax
StringFunctions
gawksplitrecords
SingleCharacterFields
EscapeSequences
Bugs
TypingandComparison
ExpressionPatterns
ComparisonOperators
POSIX/GNU
SpecialCaveats
Nondecimalnumbers
Nondecimalnumbers
Options
BasicHighLevel
Glossary
Cygwin
PCCompiling
VMSCompilation
PCCompiling
BitwiseFunctions
BitwiseFunctions
Statements
Concatenation
BreakpointControl
ConditionalExp
RegexpUsage
NondecimalData
ScalarConstants
Constants
ContinueStatement
428/479
29/7/2016
TheGNUAwkUsersGuide
controlstatements:
controllingarrayscanningorder:
convertstringtolowercase:
convertstringtonumber:
convertstringtouppercase:
convertingintegerarraysubscripts:
converting,datestotimestamps:
converting,numberstostrings:
converting,numberstostrings:
converting,stringstonumbers:
converting,stringstonumbers:
CONVFMTvariable:
CONVFMTvariable:
CONVFMTvariable,andarraysubscripts:
cookie:
coprocesses:
coprocesses:
coprocesses,closing:
coprocesses,getlinefrom:
cos()function:
cosine:
counting:
cshutility:
cshutility,POSIXLY_CORRECTenvironmentvariable:
cshutility,|&operator,comparisonwith:
ctime()userdefinedfunction:
currencysymbols,localization:
currentsystemtime:
custom.hfile:
customizedinputparser:
customizedoutputwrapper:
customizedtwowayprocessor:
cututility:
cututility:
cut.awkprogram:
Statements
ControllingScanning
StringFunctions
StringFunctions
StringFunctions
TimeFunctions
StringsAndNumbers
BitwiseFunctions
StringsAndNumbers
BitwiseFunctions
StringsAndNumbers
Usermodified
Glossary
Redirection
TwowayI/O
CloseFilesAndPipes
Getline/Coprocess
NumericFunctions
NumericFunctions
WcProgram
Statements/Lines
Options
TwowayI/O
FunctionExample
Explaininggettext
TimeFunctions
ConfigurationPhilosophy
InputParsers
OutputWrappers
Twowayprocessors
CutProgram
CutProgram
CutProgram
ddebuggercommand(aliasfordelete):
BreakpointControl
Conventions
Conventions
Glossary
TruthValues
AssignmentOps
BreakStatement
CloseFilesAndPipes
AssignmentOptions
D
d.c.,Seedarkcorner:
darkcorner:
darkcorner:
darkcorner,"0"isactuallytrue:
darkcorner,/=operatorvs./=/regexpconstant:
darkcorner,arraysubscripts:
darkcorner,breakstatement:
darkcorner,close()function:
darkcorner,commandlinearguments:
429/479
29/7/2016
TheGNUAwkUsersGuide
darkcorner,continuestatement:
darkcorner,CONVFMTvariable:
darkcorner,escapesequences:
darkcorner,escapesequences,formetacharacters:
darkcorner,exitstatement:
darkcorner,fieldseparators:
darkcorner,FILENAMEvariable:
darkcorner,FILENAMEvariable:
darkcorner,FNR/NRvariables:
darkcorner,formatcontrolcharacters:
darkcorner,formatcontrolcharacters:
darkcorner,FSasnullstring:
darkcorner,inputfiles:
darkcorner,invokingawk:
darkcorner,length()function:
darkcorner,localesdecimalpointcharacter:
darkcorner,multilinerecords:
darkcorner,NFvariable,decrementing:
darkcorner,OFMTvariable:
darkcorner,regexpassecondargumenttoindex():
darkcorner,regexpconstants:
darkcorner,regexpconstants,/=operatorand:
darkcorner,regexpconstants,asargumentstouserdefinedfunctions:
darkcorner,split()function:
darkcorner,strings,storing:
darkcorner,valueofARGV[0]:
darkcorner,^,inFS:
data,fixedwidth:
datadrivenlanguages:
database,group,reading:
database,users,reading:
dateutility,GNU:
dateutility,POSIX:
dates,convertingtotimestamps:
dates,informationrelatedto,localization:
Davies,Stephen:
Davies,Stephen:
Day,RobertP.J.:
dcgettext()function(gawk):
dcgettext()function(gawk):
dcgettext()function(gawk),portabilityand:
dcngettext()function(gawk):
dcngettext()function(gawk):
dcngettext()function(gawk),portabilityand:
deadlocks:
debuggercommands,b(break):
ContinueStatement
StringsAndNumbers
OtherArguments
EscapeSequences
ExitStatement
FullLineFields
GetlineNotes
Autoset
Autoset
ControlLetters
ControlLetters
awksplitrecords
CommandLine
StringFunctions
Localeinfluencesconversions
MultipleLine
ChangingFields
OFMT
StringFunctions
UsingConstantRegexps
AssignmentOps
StringFunctions
gawksplitrecords
Autoset
ConstantSize
BasicHighLevel
GroupFunctions
PasswdFunctions
TimeFunctions
TimeFunctions
TimeFunctions
Explaininggettext
Acknowledgments
Contributors
Acknowledgments
I18NFunctions
Programmeri18n
I18NPortability
I18NFunctions
Programmeri18n
I18NPortability
TwowayI/O
BreakpointControl
430/479
29/7/2016
debuggercommands,backtrace:
debuggercommands,break:
debuggercommands,bt(backtrace):
debuggercommands,c(continue):
debuggercommands,clear:
debuggercommands,commands:
debuggercommands,condition:
debuggercommands,continue:
debuggercommands,d(delete):
debuggercommands,delete:
debuggercommands,disable:
debuggercommands,display:
debuggercommands,down:
debuggercommands,dump:
debuggercommands,e(enable):
debuggercommands,enable:
debuggercommands,end:
debuggercommands,eval:
debuggercommands,f(frame):
debuggercommands,finish:
debuggercommands,frame:
debuggercommands,h(help):
debuggercommands,help:
debuggercommands,i(info):
debuggercommands,ignore:
debuggercommands,info:
debuggercommands,l(list):
debuggercommands,list:
debuggercommands,n(next):
debuggercommands,next:
debuggercommands,nexti:
debuggercommands,ni(nexti):
debuggercommands,o(option):
debuggercommands,option:
debuggercommands,p(print):
debuggercommands,print:
debuggercommands,printf:
debuggercommands,q(quit):
debuggercommands,quit:
debuggercommands,r(run):
debuggercommands,return:
debuggercommands,run:
debuggercommands,s(step):
debuggercommands,set:
debuggercommands,si(stepi):
debuggercommands,silent:
debuggercommands,step:
TheGNUAwkUsersGuide
ExecutionStack
BreakpointControl
ExecutionStack
BreakpointControl
BreakpointControl
BreakpointControl
BreakpointControl
BreakpointControl
ExecutionStack
MiscellaneousDebuggerCommands
BreakpointControl
BreakpointControl
ExecutionStack
ExecutionStack
DebuggerInfo
BreakpointControl
DebuggerInfo
DebuggerInfo
DebuggerInfo
431/479
29/7/2016
TheGNUAwkUsersGuide
debuggercommands,stepi:
debuggercommands,t(tbreak):
debuggercommands,tbreak:
debuggercommands,trace:
debuggercommands,u(until):
debuggercommands,undisplay:
debuggercommands,until:
debuggercommands,unwatch:
debuggercommands,up:
debuggercommands,w(watch):
debuggercommands,watch:
debuggercommands,where(backtrace):
debuggerdefaultlistamount:
debuggerhistoryfile:
debuggerhistorysize:
debuggeroptions:
debuggerprompt:
debugger,howtostart:
debugger,readcommandsfromafile:
debuggingawkprograms:
debugginggawk,bugreports:
decimalpointcharacter,localespecific:
decrementoperators:
defaultkeyword:
Deifik,Scott:
Deifik,Scott:
Deifik,Scott:
deletearray:
deletebreakpointatlocation:
deletebreakpointbynumber:
deletedebuggercommand:
deletestatement:
deletewatchpoint:
deletingelementsinarrays:
deletingentirearrays:
Demaille,Akim:
describecallstackframe,indebugger:
differencesbetweengawkandawk:
differencesinawkandgawk,ARGC/ARGVvariables:
differencesinawkandgawk,ARGINDvariable:
differencesinawkandgawk,arrayelements,deleting:
differencesinawkandgawk,AWKLIBPATHenvironmentvariable:
differencesinawkandgawk,AWKPATHenvironmentvariable:
differencesinawkandgawk,BEGIN/ENDpatterns:
differencesinawkandgawk,BEGINFILE/ENDFILEpatterns:
differencesinawkandgawk,BINMODEvariable:
differencesinawkandgawk,BINMODEvariable:
BreakpointControl
BreakpointControl
ExecutionStack
ExecutionStack
DebuggerInfo
DebuggerInfo
DebuggerInfo
DebuggerInfo
DebuggerInfo
DebuggerInvocation
DebuggerInfo
Debugger
Bugs
Options
IncrementOps
SwitchStatement
Acknowledgments
Contributors
Bugs
Delete
BreakpointControl
BreakpointControl
BreakpointControl
Delete
Delete
Delete
Acknowledgments
DebuggerInfo
StringFunctions
ARGCandARGV
Autoset
Delete
AWKLIBPATHVariable
AWKPATHVariable
I/OAndBEGIN/END
BEGINFILE/ENDFILE
Usermodified
PCUsing
432/479
29/7/2016
TheGNUAwkUsersGuide
differencesinawkandgawk,close()function:
differencesinawkandgawk,close()function:
differencesinawkandgawk,commandlinedirectories:
differencesinawkandgawk,ERRNOvariable:
differencesinawkandgawk,errormessages:
differencesinawkandgawk,FIELDWIDTHSvariable:
differencesinawkandgawk,FPATvariable:
differencesinawkandgawk,FUNCTABvariable:
differencesinawkandgawk,functionarguments(gawk):
differencesinawkandgawk,getlinecommand:
differencesinawkandgawk,IGNORECASEvariable:
differencesinawkandgawk,implementationlimitations:
differencesinawkandgawk,implementationlimitations:
differencesinawkandgawk,indirectfunctioncalls:
differencesinawkandgawk,input/outputoperators:
differencesinawkandgawk,input/outputoperators:
differencesinawkandgawk,linecontinuations:
differencesinawkandgawk,LINTvariable:
differencesinawkandgawk,match()function:
differencesinawkandgawk,print/printfstatements:
differencesinawkandgawk,PROCINFOarray:
differencesinawkandgawk,readtimeouts:
differencesinawkandgawk,recordseparators:
differencesinawkandgawk,regexpconstants:
differencesinawkandgawk,regularexpressions:
differencesinawkandgawk,RS/RTvariables:
differencesinawkandgawk,RTvariable:
differencesinawkandgawk,singlecharacterfields:
differencesinawkandgawk,split()function:
differencesinawkandgawk,strings:
differencesinawkandgawk,strings,storing:
differencesinawkandgawk,SYMTABvariable:
differencesinawkandgawk,TEXTDOMAINvariable:
differencesinawkandgawk,truncmodoperation:
directories,commandline:
directories,searching:
directories,searchingforloadableextensions:
directories,searchingforsourcefiles:
disablebreakpoint:
disabledebuggercommand:
displaydebuggercommand:
displaydebuggeroptions:
division:
dowhilestatement:
dowhilestatement,useofregexpsin:
documentation,ofawkprograms:
CloseFilesAndPipes
CloseFilesAndPipes
Autoset
SpecialFD
Usermodified
Usermodified
Autoset
CallingBuiltin
Getline
Usermodified
GetlineNotes
Redirection
IndirectCalls
Getline/Coprocess
Redirection
ConditionalExp
Usermodified
StringFunctions
FormatModifiers
Autoset
ReadTimeout
awksplitrecords
Casesensitivity
gawksplitrecords
Autoset
StringFunctions
ScalarConstants
gawksplitrecords
Autoset
Usermodified
ArithmeticOps
ProgramsExercises
AWKLIBPATHVariable
AWKPATHVariable
BreakpointControl
BreakpointControl
DebuggerInfo
ArithmeticOps
DoStatement
RegexpUsage
LibraryNames
433/479
29/7/2016
TheGNUAwkUsersGuide
documentation,online:
documents,searching:
dollarsign($),$fieldoperator:
dollarsign($),$fieldoperator:
dollarsign($),incrementingfieldsandarrays:
dollarsign($),regexpoperator:
doublequote("),inregexpconstants:
doublequote("),inshellcommands:
downdebuggercommand:
Drepper,Ulrich:
Duman,Patrice:
dumpallvariablesofaprogram:
dumpdebuggercommand:
dupword.awkprogram:
dynamicprofiling:
dynamicallyloadedextensions:
ManualHistory
DupwordProgram
Fields
Precedence
IncrementOps
RegexpOperators
ComputedRegexps
Quoting
ExecutionStack
Acknowledgments
Acknowledgments
Options
DupwordProgram
Profiling
DynamicExtensions
edebuggercommand(aliasforenable):
BreakpointControl
OrdinalFunctions
Autoset
Autoset
BracketExpressions
EgrepProgram
EgrepProgram
AssigningElements
Delete
ScanninganArray
ScanninganArray
ReferencetoElements
Bugs
TCP/IPNetworking
ReferencetoElements
Empty
awksplitrecords
BreakpointControl
BreakpointControl
BEGIN/END
UsingBEGIN/END
Profiling
AssertFunction
ExpressionPatterns
ExitStatement
I/OAndBEGIN/END
NextStatement
E
EBCDIC:
effectivegroupIDofgawkuser:
effectiveuserIDofgawkuser:
egreputility:
egreputility:
egrep.awkprogram:
elementsinarrays,assigningvalues:
elementsinarrays,deleting:
elementsinarrays,orderofaccessbyinoperator:
elementsinarrays,scanning:
elementsofarrays:
emailaddressforbugreports,buggawk@gnu.org:
EMISTERED:
emptyarrayelements:
emptypattern:
emptystrings:
emptystrings,Seenullstrings:
enablebreakpoint:
enabledebuggercommand:
enddebuggercommand:
ENDpattern:
ENDpattern:
ENDpattern,andprofiling:
ENDpattern,assert()userdefinedfunctionand:
ENDpattern,Booleanpatternsand:
ENDpattern,exitstatementand:
ENDpattern,next/nextfilestatementsand:
ENDpattern,next/nextfilestatementsand:
434/479
29/7/2016
TheGNUAwkUsersGuide
ENDpattern,operatorsand:
ENDpattern,printstatementand:
ENDFILEpattern:
ENDFILEpattern,Booleanpatternsand:
endfile()userdefinedfunction:
endgrent()function(Clibrary):
endgrent()userdefinedfunction:
endpwent()function(Clibrary):
endpwent()userdefinedfunction:
English,Steve:
ENVIRONarray:
environmentvariablesusedbygawk:
environmentvariables,inENVIRONarray:
epoch,definitionof:
equalssign(=),=operator:
equalssign(=),==operator:
equalssign(=),==operator:
EREs(ExtendedRegularExpressions):
ERRNOvariable:
ERRNOvariable:
ERRNOvariable,withBEGINFILEpattern:
ERRNOvariable,withclose()function:
ERRNOvariable,withgetlinecommand:
errorhandling:
errorhandling,ERRNOvariableand:
erroroutput:
escapeprocessing,gsub()/gensub()/sub()functions:
escapesequences,instrings:
evaldebuggercommand:
evaluateexpressions,indebugger:
evaluationorder:
evaluationorder,concatenation:
evaluationorder,functions:
examiningfields:
exclamationpoint(!),!operator:
exclamationpoint(!),!=operator:
exclamationpoint(!),!=operator:
exclamationpoint(!),!~operator:
UsingBEGIN/END
I/OAndBEGIN/END
BEGINFILE/ENDFILE
ExpressionPatterns
FiletransFunction
GroupFunctions
GroupFunctions
PasswdFunctions
PasswdFunctions
AdvancedFeatures
Autoset
EnvironmentVariables
Autoset
Glossary
AssignmentOps
ComparisonOperators
Precedence
BracketExpressions
Autoset
TCP/IPNetworking
BEGINFILE/ENDFILE
CloseFilesAndPipes
Getline
SpecialFD
Autoset
SpecialFD
GoryDetails
EscapeSequences
IncrementOps
Concatenation
CallingBuiltin
Fields
BooleanOps
Precedence
EgrepProgram
ComparisonOperators
Precedence
RegexpUsage
ComputedRegexps
Casesensitivity
RegexpConstants
ComparisonOperators
ComparisonOperators
Precedence
ExpressionPatterns
435/479
29/7/2016
TheGNUAwkUsersGuide
exitstatement:
exitstatus,ofgawk:
exitstatus,ofVMS:
exitthedebugger:
exp()function:
expandutility:
ExpatXMLparserlibrary:
exponent:
expressions:
expressions,aspatterns:
expressions,assignment:
expressions,Boolean:
expressions,comparison:
expressions,conditional:
expressions,matching,Seecomparisonexpressions:
expressions,selecting:
ExtendedRegularExpressions(EREs):
extensionAPI:
extensionAPIinformationalvariables:
extensionAPIversion:
extensionAPI,versionnumber:
extensionexample:
extensionregistration:
extensionsearchpath:
extensionsdistributedwithgawk:
extensions,allocatingmemory:
extensions,BrianKernighansawk:
extensions,BrianKernighansawk:
extensions,common,**operator:
extensions,common,**=operator:
extensions,common,/dev/stderrspecialfile:
extensions,common,/dev/stdinspecialfile:
extensions,common,/dev/stdoutspecialfile:
extensions,common,BINMODEvariable:
extensions,common,deletetodeleteentirearrays:
extensions,common,fflush()function:
extensions,common,funckeyword:
extensions,common,length()appliedtoanarray:
extensions,common,RSasaregexp:
extensions,common,singlecharacterfields:
extensions,common,\xescapesequence:
extensions,ingawk,notinPOSIXawk:
extensions,loading,@loaddirective:
extensions,mawk:
extensions,wheretofind:
extract.awkprogram:
extraction,ofmarkedstrings(internationalization):
ExitStatement
ExitStatus
VMSRunning
NumericFunctions
VerySimple
gawkextlib
NumericFunctions
Expressions
ExpressionPatterns
AssignmentOps
BooleanOps
TypingandComparison
ConditionalExp
TypingandComparison
ConditionalExp
BracketExpressions
ExtensionAPIDescription
ExtensionAPIInformationalVariables
ExtensionVersioning
Autoset
ExtensionExample
RegistrationFunctions
FindingExtensions
ExtensionSamples
MemoryAllocationFunctions
BTL
CommonExtensions
ArithmeticOps
AssignmentOps
SpecialFD
SpecialFD
SpecialFD
PCUsing
Delete
I/OFunctions
DefinitionSyntax
StringFunctions
gawksplitrecords
EscapeSequences
POSIX/GNU
CommonExtensions
gawkextlib
ExtractProgram
StringExtraction
436/479
29/7/2016
TheGNUAwkUsersGuide
F
fdebuggercommand(aliasforframe):
false,logical:
FDL(FreeDocumentationLicense):
features,addingtogawk:
features,deprecated:
features,undocumented:
Fenlason,Jay:
Fenlason,Jay:
fflush()function:
fieldnumbers:
fieldoperator$:
fieldoperators,dollarsignas:
fieldseparator,inmultilinerecords:
fieldseparator,oncommandline:
fieldseparator,POSIXand:
fieldseparators:
fieldseparators:
fieldseparators:
fieldseparators,choiceof:
fieldseparators,FIELDWIDTHSvariableand:
fieldseparators,FPATvariableand:
fieldseparators,POSIXand:
fieldseparators,regularexpressionsas:
fieldseparators,regularexpressionsas:
fieldseparators,SeeAlsoOFS:
fieldseparators,spacesas:
fields:
fields:
fields:
fields,adding:
fields,changingcontentsof:
fields,cutting:
fields,examining:
fields,numberof:
fields,numbers:
fields,printing:
fields,separating:
fields,separating:
fields,singlecharacter:
FIELDWIDTHSvariable:
FIELDWIDTHSvariable:
filedescriptors:
fileinclusion,@includedirective:
filenames,distinguishing:
filenames,incompatibilitymode:
ExecutionStack
TruthValues
AddingCode
Obsolete
Undocumented
History
Contributors
I/OFunctions
NonconstantFields
Fields
Fields
MultipleLine
FullLineFields
FieldSeparators
Usermodified
Usermodified
FieldSeparators
Usermodified
Usermodified
Fields
FieldSeparators
ChangingFields
CutProgram
ReadingFiles
Fields
BasicHighLevel
ChangingFields
ChangingFields
CutProgram
Fields
Fields
NonconstantFields
PrintExamples
FieldSeparators
FieldSeparators
ConstantSize
Usermodified
SpecialFD
IncludeFiles
Autoset
SpecialCaveats
437/479
29/7/2016
TheGNUAwkUsersGuide
filenames,standardstreamsingawk:
FILENAMEvariable:
FILENAMEvariable:
FILENAMEvariable,getline,settingwith:
filenames,assignmentsas:
files,.gmo:
files,.gmo,specifyingdirectoryof:
files,.gmo,specifyingdirectoryof:
files,.mo,convertingfrom.po:
files,.po:
files,.po:
files,.po,convertingto.mo:
files,.pot:
files,/dev/specialfiles:
files,/inet/(gawk):
files,/inet4/(gawk):
files,/inet6/(gawk):
files,awkprogramsin:
files,awkprof.out:
files,awkvars.out:
files,closing:
files,descriptors,Seefiledescriptors:
files,group:
files,initializationandcleanup:
files,input,Seeinputfiles:
files,log,timestampsin:
files,managing:
files,managing,datafileboundaries:
files,messageobject:
files,messageobject,convertingfromportableobjectfiles:
files,messageobject,specifyingdirectoryof:
files,messageobject,specifyingdirectoryof:
files,multiplepassesover:
files,multiple,duplicatingoutputinto:
files,output,Seeoutputfiles:
files,password:
files,portableobject:
files,portableobject:
files,portableobjecttemplate:
files,portableobject,convertingtomessageobjectfiles:
files,portableobject,generating:
files,processing,ARGINDvariableand:
files,reading:
SpecialFD
ReadingFiles
Autoset
GetlineNotes
IgnoringAssigns
Explaininggettext
Explaininggettext
Programmeri18n
I18NExample
Explaininggettext
Translatori18n
I18NExample
Explaininggettext
SpecialFD
TCP/IPNetworking
TCP/IPNetworking
TCP/IPNetworking
Long
Profiling
Options
I/OFunctions
SpecialFD
GroupFunctions
FiletransFunction
ReadTerminal
TimeFunctions
DataFileManagement
FiletransFunction
Explaininggettext
I18NExample
Explaininggettext
Programmeri18n
OtherArguments
TeeProgram
CloseFilesAndPipes
PasswdFunctions
Explaininggettext
Translatori18n
Explaininggettext
I18NExample
Options
Autoset
RewindFunction
files,reading,multilinerecords:
files,searchingforregularexpressions:
files,skipping:
files,source,searchpathfor:
MultipleLine
EgrepProgram
FileChecking
ProgramsExercises
438/479
29/7/2016
TheGNUAwkUsersGuide
files,splitting:
files,Texinfo,extractingprogramsfrom:
findsubstringinstring:
findingextensions:
finishdebuggercommand:
Fish,Fred:
fixedwidthdata:
flagvariables:
flagvariables:
floatingpoint,numbers,arbitraryprecision:
floatingpoint,VAX/VMS:
flushbufferedoutput:
fnmatch()extensionfunction:
FNRvariable:
FNRvariable:
FNRvariable,changing:
forstatement:
forstatement,loopingoverarrays:
fork()extensionfunction:
formatspecifiers:
formatspecifiers,mixingregularwithpositionalspecifiers:
formatspecifiers,printfstatement:
formatspecifiers,strftime()function(gawk):
formattimestring:
formats,numericoutput:
formattingoutput:
formattingstrings:
forwardslash(/)toencloseregularexpressions:
forwardslash(/),/operator:
forwardslash(/),/=operator:
forwardslash(/),/=operator:
forwardslash(/),/=operator,vs./=/regexpconstant:
forwardslash(/),patternsand:
FPATvariable:
FPATvariable:
framedebuggercommand:
FreeDocumentationLicense(FDL):
FreeSoftwareFoundation(FSF):
FreeBSD:
SplitProgram
ExtractProgram
StringFunctions
FindingExtensions
Contributors
ConstantSize
BooleanOps
TeeProgram
VMSRunning
I/OFunctions
ExtensionSampleFnmatch
Records
Autoset
Autoset
ForStatement
ScanninganArray
ExtensionSampleFork
BasicPrintf
PrintfOrdering
ControlLetters
TimeFunctions
TimeFunctions
OFMT
Printf
StringFunctions
Regexp
Precedence
AssignmentOps
Precedence
AssignmentOps
ExpressionPatterns
SplittingByContent
Usermodified
ExecutionStack
ManualHistory
Getting
Glossary
Glossary
Glossary
FSvariable:
FieldSeparators
Usermodified
Options
FSvariable:
FSvariable,fieldseparatoroptionand:
FSvariable,asnullstring:
439/479
29/7/2016
TheGNUAwkUsersGuide
FSvariable,asTABcharacter:
FSF(FreeSoftwareFoundation):
fts()extensionfunction:
FUNCTABarray:
functioncalls:
functioncalls,indirect:
functioncalls,indirect,@notationfor:
functiondefinitionexample:
functionpointers:
functions,arraysasparametersto:
functions,builtin:
functions,builtin:
functions,builtin,evaluationorder:
functions,defining:
functions,library:
functions,library,assertions:
functions,library,associativearraysand:
functions,library,Clibrary:
functions,library,charactervaluesasnumbers:
functions,library,Cliffrandomnumbers:
functions,library,commandlineoptions:
functions,library,exampleprogramforusing:
functions,library,groupdatabase,reading:
functions,library,managingdatafiles:
functions,library,managingtime:
functions,library,mergingarraysintostrings:
functions,library,roundingnumbers:
functions,library,userdatabase,reading:
functions,namesof:
functions,recursive:
functions,stringtranslation:
functions,undefined:
functions,userdefined:
functions,userdefined,calling:
functions,userdefined,counts,inaprofile:
Options
FieldSeparators
CutProgram
MultipleLine
ManualHistory
Getting
Glossary
Glossary
Autoset
FunctionCalls
IndirectCalls
IndirectCalls
FunctionExample
IndirectCalls
FunctionCalls
Functions
CallingBuiltin
DefinitionSyntax
LibraryFunctions
AssertFunction
LibraryNames
GetoptFunction
OrdinalFunctions
CliffRandomFunction
GetoptFunction
IgawkProgram
GroupFunctions
DataFileManagement
GetlocaltimeFunction
JoinFunction
RoundFunction
PasswdFunctions
DefinitionSyntax
DefinitionSyntax
I18NFunctions
Userdefined
FunctionCaveats
Profiling
functions,userdefined,libraryof:
functions,userdefined,next/nextfilestatementsand:
functions,userdefined,next/nextfilestatementsand:
LibraryFunctions
NextStatement
NextfileStatement
FSvariable,changingvalueof:
FSvariable,runningawkprogramsand:
FSvariable,settingfromcommandline:
FS,containing^:
FS,inmultilinerecords:
440/479
29/7/2016
TheGNUAwkUsersGuide
G
Gd:
Garfinkle,Scott:
gawkprogram,dynamicprofiling:
gawkversion:
gawk,ARGINDvariablein:
gawk,awkand:
gawk,awkand:
gawk,bitwiseoperationsin:
gawk,breakstatementin:
gawk,characterclassesand:
gawk,codingstylein:
gawk,commandlineoptions,andregularexpressions:
gawk,configuring:
gawk,configuring,options:
gawk,continuestatementin:
gawk,distribution:
gawk,ERRNOvariablein:
gawk,escapesequences:
gawk,extensions,disabling:
gawk,features,adding:
gawk,features,advanced:
gawk,fieldseparatorsand:
gawk,FIELDWIDTHSvariablein:
gawk,FIELDWIDTHSvariablein:
gawk,filenamesin:
gawk,formatcontrolcharacters:
gawk,formatcontrolcharacters:
gawk,FPATvariablein:
gawk,FPATvariablein:
gawk,FUNCTABarrayin:
gawk,functionargumentsand:
gawk,hexadecimalnumbersand:
gawk,IGNORECASEvariablein:
gawk,implementationissues:
Acknowledgments
Contributors
Profiling
Autoset
OtherArguments
Preface
ThisManual
BitwiseFunctions
BreakStatement
BracketExpressions
AddingCode
GNURegexpOperators
ConfigurationPhilosophy
ContinueStatement
Getline
CloseFilesAndPipes
BEGINFILE/ENDFILE
Autoset
TCP/IPNetworking
EscapeSequences
Options
AddingCode
AdvancedFeatures
Usermodified
ConstantSize
Usermodified
SpecialFiles
ControlLetters
ControlLetters
SplittingByContent
Usermodified
Autoset
CallingBuiltin
Nondecimalnumbers
Casesensitivity
Usermodified
ArrayIntro
StringFunctions
Notes
gawk,implementationissues,debugging:
CompatibilityMode
CompatibilityMode
GetlineNotes
gawk,implementationissues,downwardcompatibility:
gawk,implementationissues,limits:
441/479
29/7/2016
TheGNUAwkUsersGuide
gawk,implementationissues,pipes:
gawk,installing:
gawk,internationalizationand,Seeinternationalization:
gawk,interpreter,addingcodeto:
gawk,intervalexpressionsand:
gawk,linecontinuationin:
gawk,LINTvariablein:
gawk,listofcontributorsto:
gawk,MSDOSversionof:
gawk,MSWindowsversionof:
gawk,newlinesin:
gawk,octalnumbersand:
gawk,OS/2versionof:
gawk,predefinedvariablesand:
gawk,PROCINFOarrayin:
gawk,regexpconstantsand:
gawk,regularexpressions,casesensitivity:
gawk,regularexpressions,operators:
gawk,regularexpressions,precedence:
gawk,RTvariablein:
gawk,RTvariablein:
gawk,RTvariablein:
gawk,SeeAlsoawk:
gawk,sourcecode,obtaining:
gawk,splittingfieldsand:
gawk,stringtranslationfunctions:
gawk,SYMTABarrayin:
gawk,TEXTDOMAINvariablein:
gawk,timestamps:
gawk,usesfor:
gawk,versionsof,informationabout,printing:
gawk,VMSversionof:
gawk,wordboundaryoperator:
gawkextlib:
gawkextlibproject:
GeneralPublicLicense(GPL):
GeneralPublicLicense,SeeGPL:
generatetimevalues:
gensub()function(gawk):
gensub()function(gawk):
gensub()function(gawk),escapeprocessing:
getaddrinfo()function(Clibrary):
getgrent()function(Clibrary):
getgrent()function(Clibrary):
Redirection
Installation
UsingInternalFileOps
RegexpOperators
ConditionalExp
Usermodified
Contributors
PCUsing
PCUsing
Statements/Lines
Nondecimalnumbers
PCUsing
BuiltinVariables
Autoset
Autoset
TimeFunctions
TwowayI/O
Casesensitivity
GNURegexpOperators
RegexpOperators
awksplitrecords
MultipleLine
Autoset
Preface
Getting
ConstantSize
I18NFunctions
Autoset
Usermodified
TimeFunctions
Preface
Options
VMSInstallation
GNURegexpOperators
gawkextlib
gawkextlib
Glossary
ManualHistory
TimeFunctions
StringFunctions
GoryDetails
TCP/IPNetworking
GroupFunctions
GroupFunctions
442/479
29/7/2016
TheGNUAwkUsersGuide
getgrent()userdefinedfunction:
Git,useofforgawksourcecode:
GNITSmailinglist:
GNUawk,Seegawk:
GroupFunctions
GroupFunctions
GroupFunctions
GroupFunctions
GroupFunctions
GroupFunctions
GroupFunctions
GroupFunctions
ReadingFiles
Getline/Coprocess
CloseFilesAndPipes
TwowayI/O
Getline
GetlineNotes
Getline
GetlineSummary
GroupFunctions
PasswdFunctions
Getline/File
Getline/Variable
BEGINFILE/ENDFILE
GetoptFunction
GetoptFunction
GetoptFunction
PasswdFunctions
PasswdFunctions
PasswdFunctions
PasswdFunctions
PasswdFunctions
PasswdFunctions
PasswdFunctions
PasswdFunctions
Explaininggettext
Explaininggettext
Explaininggettext
ExtensionSampleTime
gawkextlib
OtherVersions
AccessingTheSource
AddingCode
DerivedFiles
Acknowledgments
Preface
GNUFreeDocumentationLicense:
GNUGeneralPublicLicense:
Glossary
getgrent()userdefinedfunction:
getgrgid()function(Clibrary):
getgrgid()userdefinedfunction:
getgrnam()function(Clibrary):
getgrnam()userdefinedfunction:
getgruser()function(Clibrary):
getgruser()function,userdefined:
getlinecommand:
getlinecommand,coprocesses,usingfrom:
getlinecommand,coprocesses,usingfrom:
getlinecommand,deadlockand:
getlinecommand,explicitinputwith:
getlinecommand,FILENAMEvariableand:
getlinecommand,returnvalues:
getlinecommand,variants:
getlinecommand,_gr_init()userdefinedfunction:
getlinecommand,_pw_init()function:
getlinefromafile:
getlineintoavariable:
getlinestatement,BEGINFILE/ENDFILEpatternsand:
getlocaltime()userdefinedfunction:
getopt()function(Clibrary):
getopt()userdefinedfunction:
getopt()userdefinedfunction:
getpwent()function(Clibrary):
getpwent()function(Clibrary):
getpwent()userdefinedfunction:
getpwent()userdefinedfunction:
getpwnam()function(Clibrary):
getpwnam()userdefinedfunction:
getpwuid()function(Clibrary):
getpwuid()userdefinedfunction:
gettextlibrary:
gettextlibrary,localecategories:
gettext()function(Clibrary):
gettimeofday()extensionfunction:
gitutility:
gitutility:
gitutility:
gitutility:
443/479
29/7/2016
TheGNUAwkUsersGuide
GNULesserGeneralPublicLicense:
GNUlongoptions:
GNUlongoptions:
GNUlongoptions,printinglistof:
GNUProject:
GNUProject:
GNU/Linux:
GNU/Linux:
GNU/Linux:
Gordon,Assaf:
GPL(GeneralPublicLicense):
GPL(GeneralPublicLicense):
GPL(GeneralPublicLicense),printing:
grcatprogram:
Grigera,Juan:
groupdatabase,reading:
groupfile:
groupIDofgawkuser:
groups,informationabout:
gsub()function:
gsub()function:
gsub()function,argumentsof:
gsub()function,escapeprocessing:
Glossary
CommandLine
Options
Options
ManualHistory
Glossary
ManualHistory
I18NExample
Glossary
Contributors
ManualHistory
Glossary
Options
GroupFunctions
Contributors
GroupFunctions
GroupFunctions
Autoset
GroupFunctions
StringFunctions
StringFunctions
GoryDetails
hdebuggercommand(aliasforhelp):
Hankerson,Darrel:
Hankerson,Darrel:
Haque,John:
Hartholz,Elaine:
Hartholz,Marshall:
Hasegawa,Isamu:
helpdebuggercommand:
hexadecimalnumbers:
hexadecimalvalues,enablinginterpretationof:
historyexpansion,indebugger:
histsort.awkprogram:
Hughes,Phil:
HUPsignal,fordynamicprofiling:
hyphen(),operator:
hyphen(),operator:
hyphen(),operator:
hyphen(),operator:
hyphen(),=operator:
hyphen(),=operator:
Acknowledgments
Contributors
Contributors
Acknowledgments
Acknowledgments
Contributors
Nondecimalnumbers
Options
ReadlineSupport
HistorySorting
Acknowledgments
Profiling
Precedence
Precedence
IncrementOps
Precedence
AssignmentOps
Precedence
hyphen(),filenamesbeginningwith:
hyphen(),inbracketexpressions:
Options
BracketExpressions
444/479
29/7/2016
TheGNUAwkUsersGuide
I
idebuggercommand(aliasforinfo):
ignorebreakpoint:
ignoredebuggercommand:
IGNORECASEvariable:
IGNORECASEvariable,andarrayindices:
IGNORECASEvariable,andarraysortingfunctions:
IGNORECASEvariable,inexampleprograms:
IGNORECASEvariable,with~and!~operators:
Illumos:
Illumos,POSIXcompliantawk:
implementationissues,gawk:
implementationissues,gawk,debugging:
implementationissues,gawk,limits:
implementationissues,gawk,limits:
inoperator:
inoperator:
inoperator:
inoperator,indexexistenceinmultidimensionalarrays:
inoperator,orderofarrayaccess:
inoperator,testingifarrayelementexists:
inoperator,useinloops:
includingfiles,@includedirective:
incrementoperators:
index()function:
indexingarrays:
indirectfunctioncalls:
indirectfunctioncalls,@notation:
infiniteprecision:
infodebuggercommand:
initialization,automatic:
inplaceextension:
inputfiles:
inputfiles,closing:
inputfiles,countingelementsin:
inputfiles,examples:
inputfiles,reading:
inputfiles,runningawkwithout:
DebuggerInfo
IdProgram
IdProgram
IfStatement
Ranges
RegexpUsage
IgawkProgram
BreakpointControl
BreakpointControl
Usermodified
ArrayIntro
LibraryFunctions
Casesensitivity
OtherVersions
OtherVersions
Notes
CompatibilityMode
GetlineNotes
Redirection
ComparisonOperators
Precedence
ForStatement
Multidimensional
ScanninganArray
ReferencetoElements
ScanninganArray
IncludeFiles
IncrementOps
StringFunctions
ArrayIntro
IndirectCalls
IndirectCalls
DebuggerInfo
MoreComplex
ExtensionSampleInplace
ReadingFiles
CloseFilesAndPipes
WcProgram
SampleDataFiles
ReadingFiles
ReadTerminal
inputfiles,runningawkwithout:
inputfiles,variableassignmentsand:
ReadTerminal
OtherArguments
idutility:
id.awkprogram:
ifstatement:
ifstatement,actions,changing:
ifstatement,useofregexpsin:
igawk.shprogram:
445/479
29/7/2016
TheGNUAwkUsersGuide
inputpipeline:
inputrecord,lengthof:
inputredirection:
input,data,nondecimal:
input,explicit:
input,files,Seeinputfiles:
input,multilinerecords:
input,splittingintorecords:
input,standard:
input,standard:
input/outputfunctions:
input/output,binary:
input/output,fromBEGINandEND:
input/output,twoway:
insomnia,curefor:
installation,VMS:
installinggawk:
instructiontracing,indebugger:
INTsignal(MSWindows):
int()function:
integerarrayindices:
integers,arbitraryprecision:
integers,unsigned:
interactingwithotherprograms:
internationalization:
internationalization:
internationalization,localization:
internationalization,localization:
internationalization,localization,characterclasses:
internationalization,localization,gawkand:
internationalization,localization,localecategories:
internationalization,localization,markedstrings:
internationalization,localization,portabilityand:
internationalizingaprogram:
interpretedprograms:
interpretedprograms:
intervalexpressions,regexpoperator:
inventoryshippedfile:
invokeshellcommand:
isarray()function(gawk):
ISO:
ISO88591:
ISOLatin1:
Getline/Pipe
StringFunctions
Getline/File
NondecimalData
Getline
MultipleLine
MultipleLine
Records
ReadTerminal
SpecialFD
I/OFunctions
Usermodified
I/OAndBEGIN/END
TwowayI/O
AlarmProgram
VMSInstallation
Installation
DebuggerInfo
Profiling
NumericFunctions
ArbitraryPrecisionIntegers
ComputerArithmetic
I/OFunctions
I18NFunctions
I18NandL10N
Usermodified
BracketExpressions
Explaininggettext
Programmeri18n
I18NPortability
Explaininggettext
BasicHighLevel
Glossary
RegexpOperators
SampleDataFiles
I/OFunctions
TypeFunctions
Glossary
Glossary
Glossary
Jacobs,Andrew:
Jaegermann,Michal:
PasswdFunctions
Acknowledgments
446/479
29/7/2016
TheGNUAwkUsersGuide
Jaegermann,Michal:
Javaimplementationofawk:
Javaprogramminglanguage:
jawk:
Jediknights:
Johansen,Chris:
join()userdefinedfunction:
Contributors
OtherVersions
Glossary
OtherVersions
Undocumented
SignatureProgram
JoinFunction
Kahrs,Jrgen:
Kahrs,Jrgen:
Kasal,Stepan:
Kenobi,ObiWan:
Kernighan,Brian:
Kernighan,Brian:
Kernighan,Brian:
Kernighan,Brian:
Kernighan,Brian:
Kernighan,Brian:
Kernighan,Brian:
Kernighan,Brian:
Kernighan,Brian:
Kernighan,Brian:
Kernighan,Brian:
killcommand,dynamicprofiling:
Knights,jedi:
Kwok,Conrad:
Acknowledgments
Contributors
Acknowledgments
Undocumented
History
Conventions
Acknowledgments
Getline/Pipe
Concatenation
LibraryFunctions
BTL
Contributors
OtherVersions
BasicDataTyping
Glossary
Profiling
Undocumented
Contributors
ldebuggercommand(aliasforlist):
LabelsProgram
AdvancedFeatures
BasicHighLevel
Explaininggettext
Explaininggettext
Explaininggettext
Explaininggettext
Programmeri18n
Explaininggettext
Explaininggettext
Explaininggettext
ComparisonOperators
Precedence
Getline/File
ComparisonOperators
Precedence
BitwiseFunctions
L
labels.awkprogram:
Langston,Peter:
languages,datadriven:
LC_ALLlocalecategory:
LC_COLLATElocalecategory:
LC_CTYPElocalecategory:
LC_MESSAGESlocalecategory:
LC_MESSAGESlocalecategory,bindtextdomain()function(gawk):
LC_MONETARYlocalecategory:
LC_NUMERIClocalecategory:
LC_TIMElocalecategory:
leftanglebracket(<),<operator:
leftanglebracket(<),<operator:
leftanglebracket(<),<operator(I/O):
leftanglebracket(<),<=operator:
leftanglebracket(<),<=operator:
leftshift:
447/479
29/7/2016
TheGNUAwkUsersGuide
leftshift,bitwise:
leftmostlongestmatch:
lengthofinputrecord:
lengthofstring:
length()function:
LesserGeneralPublicLicense(LGPL):
LGPL(LesserGeneralPublicLicense):
libmawk:
librariesofawkfunctions:
librariesofawkfunctions,assertions:
librariesofawkfunctions,associativearraysand:
librariesofawkfunctions,charactervaluesasnumbers:
librariesofawkfunctions,commandlineoptions:
librariesofawkfunctions,exampleprogramforusing:
librariesofawkfunctions,groupdatabase,reading:
librariesofawkfunctions,managing,datafiles:
librariesofawkfunctions,managing,time:
librariesofawkfunctions,mergingarraysintostrings:
librariesofawkfunctions,roundingnumbers:
librariesofawkfunctions,userdatabase,reading:
linebreaks:
linecontinuations:
linecontinuations,gawk:
linecontinuations,inprintstatement:
linecontinuations,withCshell:
lines,blank,printing:
lines,counting:
lines,duplicate,removing:
lines,matchingrangesof:
lines,skippingbetweenmarkers:
lintchecking:
lintchecking,arrayelements:
lintchecking,arraysubscripts:
lintchecking,emptyprograms:
lintchecking,issuingwarnings:
lintchecking,POSIXLY_CORRECTenvironmentvariable:
lintchecking,undefinedfunctions:
LINTvariable:
Linux:
Linux:
Linux:
listallglobalvariables,indebugger:
listdebuggercommand:
listfunctiondefinitions,indebugger:
loadingextensions,@loaddirective:
BitwiseFunctions
MultipleLine
StringFunctions
StringFunctions
StringFunctions
Glossary
Glossary
OtherVersions
LibraryFunctions
AssertFunction
LibraryNames
OrdinalFunctions
GetoptFunction
IgawkProgram
GroupFunctions
DataFileManagement
JoinFunction
RoundFunction
PasswdFunctions
Statements/Lines
BooleanOps
ConditionalExp
PrintExamples
MoreComplex
Print
WcProgram
HistorySorting
Ranges
Ranges
Usermodified
Delete
CommandLine
Options
Options
Usermodified
ManualHistory
I18NExample
Glossary
DebuggerInfo
DebuggerInfo
loading,extensions:
localvariables,inafunction:
Options
VariableScope
448/479
29/7/2016
TheGNUAwkUsersGuide
localecategories:
localedecimalpointcharacter:
locale,definitionof:
localization:
localization,Seeinternationalization,localization:
logfiles,timestampsin:
log()function:
logarithm:
logicalfalse/true:
logicaloperators,SeeBooleanexpressions:
logininformation:
longoptions:
loops:
loops,breakstatementand:
loops,continuestatementsand:
loops,countforheader,inaprofile:
loops,dowhile:
loops,exiting:
loops,for,arrayscanning:
loops,for,iterative:
loops,SeeAlsowhilestatement:
loops,while:
lsutility:
lshift()function(gawk):
lvalues/rvalues:
Explaininggettext
Options
Locales
I18NandL10N
I18NandL10N
TimeFunctions
NumericFunctions
NumericFunctions
TruthValues
BooleanOps
PasswdFunctions
CommandLine
WhileStatement
BreakStatement
ForStatement
Profiling
DoStatement
BreakStatement
ScanninganArray
ForStatement
WhileStatement
WhileStatement
MoreComplex
BitwiseFunctions
AssignmentOps
maillistfile:
SampleDataFiles
LabelsProgram
Acknowledgments
Acknowledgments
Bugs
Contributors
OrdinalFunctions
StringExtraction
StringExtraction
IncrementOps
StringFunctions
StringFunctions
StringFunctions
TypingandComparison
MultipleLine
StringFunctions
EscapeSequences
Getline/Pipe
M
mailinglabels,printing:
mailinglist,GNITS:
Malmberg,John:
Malmberg,John:
Malmberg,JohnE.:
markparity:
markedstringextraction(internationalization):
markedstrings,extracting:
Marx,Groucho:
matchregexpinstring:
match()function:
match()function,RSTART/RLENGTHvariables:
matching,expressions,Seecomparisonexpressions:
matching,leftmostlongest:
matching,nullstrings:
mawkutility:
mawkutility:
mawkutility:
mawkutility:
Concatenation
NextfileStatement
449/479
29/7/2016
TheGNUAwkUsersGuide
mawkutility:
maximumprecisionsupportedbyMPFRlibrary:
McIlroy,Doug:
McPhee,Patrick:
messageobjectfiles:
messageobjectfiles,convertingfromportableobjectfiles:
messageobjectfiles,specifyingdirectoryof:
messageobjectfiles,specifyingdirectoryof:
messagesfromextensions:
metacharactersinregularexpressions:
metacharacters,escapesequencesfor:
minimumprecisionrequiredbyMPFRlibrary:
mktime()function(gawk):
modifiers,informatspecifiers:
monetaryinformation,localization:
Moore,Duncan:
msgfmtutility:
multipleprecision:
multiplelinerecords:
OtherVersions
Autoset
Glossary
Contributors
Explaininggettext
I18NExample
Explaininggettext
Programmeri18n
PrintingMessages
RegexpOperators
EscapeSequences
Autoset
TimeFunctions
FormatModifiers
Explaininggettext
GetlineNotes
I18NExample
MultipleLine
N
ndebuggercommand(aliasfornext):
names,arrays/variables:
names,functions:
names,functions:
namespaceissues:
namespaceissues,functions:
NetBSD:
networks,programming:
networks,supportfor:
newlines:
newlines:
newlines:
newlines,asfieldseparators:
newlines,asrecordseparators:
newlines,indynamicregexps:
newlines,inregexpconstants:
newlines,printing:
newlines,separatingstatementsinactions:
newlines,separatingstatementsinactions:
nextdebuggercommand:
nextfilestatement:
nextstatement:
nextstatement:
nextstatement,BEGIN/ENDpatternsand:
nextstatement,BEGINFILE/ENDFILEpatternsand:
nextstatement,userdefinedfunctionsand:
LibraryNames
DefinitionSyntax
LibraryNames
LibraryNames
DefinitionSyntax
Glossary
TCP/IPNetworking
SpecialNetwork
Statements/Lines
Options
BooleanOps
DefaultFieldSplitting
awksplitrecords
ComputedRegexps
ComputedRegexps
PrintExamples
ActionOverview
Statements
FeatureHistory
BooleanOps
NextStatement
I/OAndBEGIN/END
BEGINFILE/ENDFILE
NextStatement
450/479
29/7/2016
TheGNUAwkUsersGuide
nextfilestatement:
nextfilestatement,BEGIN/ENDpatternsand:
nextfilestatement,BEGINFILE/ENDFILEpatternsand:
nextfilestatement,userdefinedfunctionsand:
nextidebuggercommand:
NFvariable:
NFvariable:
NFvariable,decrementing:
nidebuggercommand(aliasfornexti):
noassign.awkprogram:
nonexistentarrayelements:
notBooleanlogicoperator:
NRvariable:
NRvariable:
NRvariable,changing:
nullstrings:
nullstrings:
nullstrings:
nullstrings:
nullstringsingawkarguments,quotingand:
nullstrings,anddeletingarrayelements:
nullstrings,asarraysubscripts:
nullstrings,convertingnumberstostrings:
nullstrings,matching:
numberasstringofbits:
numberofarrayelements:
numbersign(#),#!(executablescripts):
numbersign(#),commenting:
numbers,asarraysubscripts:
numbers,asvaluesofcharacters:
numbers,Cliffrandom:
numbers,converting:
numbers,converting:
numbers,converting,tostrings:
numbers,converting,tostrings:
numbers,hexadecimal:
numbers,octal:
numbers,rounding:
numericconstants:
numericfunctions:
numeric,outputformat:
numeric,strings:
NextfileStatement
I/OAndBEGIN/END
BEGINFILE/ENDFILE
NextfileStatement
Fields
Autoset
ChangingFields
IgnoringAssigns
ReferencetoElements
BooleanOps
Records
Autoset
Autoset
awksplitrecords
TruthValues
BasicDataTyping
Quoting
Delete
StringsAndNumbers
StringFunctions
BitwiseFunctions
StringFunctions
ExecutableScripts
Comments
OrdinalFunctions
CliffRandomFunction
StringsAndNumbers
BitwiseFunctions
Usermodified
Usermodified
Nondecimalnumbers
Nondecimalnumbers
RoundFunction
ScalarConstants
NumericFunctions
OFMT
VariableTyping
O
odebuggercommand(aliasforoption):
obsoletefeatures:
octalnumbers:
DebuggerInfo
Obsolete
Nondecimalnumbers
451/479
29/7/2016
TheGNUAwkUsersGuide
octalvalues,enablinginterpretationof:
Options
OFMTvariable:
OFMT
OFMTvariable:
StringsAndNumbers
OFMTvariable:
Usermodified
OFMTvariable,POSIXawkand:
OFMT
OFSvariable:
ChangingFields
OFSvariable:
OutputSeparators
OFSvariable:
Usermodified
OpenBSD:
Glossary
OpenSolaris:
OtherVersions
operatingsystems,BSDbased:
ManualHistory
operatingsystems,PC,gawkon:
PCUsing
operatingsystems,PC,gawkon,installing:
PCInstallation
operatingsystems,portinggawkto:
NewPorts
operatingsystems,SeeAlsoGNU/Linux,PCoperatingsystems,Unix: Installation
operations,bitwise:
BitwiseFunctions
operators,arithmetic:
ArithmeticOps
operators,assignment:
AssignmentOps
operators,assignment:
AssignmentOps
operators,assignment,evaluationorder:
AssignmentOps
operators,Boolean,SeeBooleanexpressions:
BooleanOps
operators,decrement/increment:
IncrementOps
operators,GNUspecific:
GNURegexpOperators
operators,input/output:
Getline/File
Getline/Pipe
Getline/Coprocess
Redirection
Redirection
Precedence
Precedence
Precedence
operators,logical,SeeBooleanexpressions:
BooleanOps
operators,precedence:
IncrementOps
operators,precedence:
Precedence
operators,relational,Seeoperators,comparison:
TypingandComparison
operators,shortcircuit:
BooleanOps
operators,string:
Concatenation
operators,stringmatching:
RegexpUsage
operators,stringmatching,forbuffers:
GNURegexpOperators
operators,wordboundary(gawk):
GNURegexpOperators
optiondebuggercommand:
DebuggerInfo
options,commandline:
Options
options,commandline,endof:
options,commandline,invokingawk:
options,commandline,processing:
options,deprecated:
Options
CommandLine
GetoptFunction
Obsolete
452/479
29/7/2016
TheGNUAwkUsersGuide
options,long:
options,long:
options,printinglistof:
ORbitwiseoperation:
orBooleanlogicoperator:
or()function(gawk):
ord()extensionfunction:
ord()userdefinedfunction:
orderofevaluation,concatenation:
ORSvariable:
ORSvariable:
outputfieldseparator,SeeOFSvariable:
outputrecordseparator,SeeORSvariable:
outputredirection:
outputwrapper:
output,buffering:
output,buffering:
output,duplicatingintofiles:
output,files,closing:
output,formatspecifier,OFMT:
output,formatted:
output,pipes:
output,printing,Seeprinting:
output,records:
output,standard:
CommandLine
Options
Options
BitwiseFunctions
BooleanOps
BitwiseFunctions
ExtensionSampleOrd
OrdinalFunctions
Concatenation
OutputSeparators
Usermodified
ChangingFields
OutputSeparators
Redirection
OutputWrappers
I/OFunctions
I/OFunctions
TeeProgram
CloseFilesAndPipes
OFMT
Printf
Redirection
Printing
OutputSeparators
SpecialFD
pdebuggercommand(aliasforprint):
Contributors
Autoset
Profiling
RegexpOperators
PasswdFunctions
StringFunctions
PatternsandActions
ExpressionPatterns
Profiling
VerySimple
Empty
RegexpPatterns
Ranges
ExpressionPatterns
PatternOverview
P
Papadopoulos,Panos:
parentprocessIDofgawkprocess:
parentheses(),inaprofile:
parentheses(),regexpoperator:
passwordfile:
patsplit()function(gawk):
patterns:
patterns,comparisonexpressionsas:
patterns,counts,inaprofile:
patterns,default:
patterns,empty:
patterns,expressionsas:
patterns,rangesin:
patterns,regexpconstantsas:
patterns,typesof:
pawk(profilingversionofBrianKernighansawk):
pawk,awklikefacilitiesforPython:
PCoperatingsystems,gawkon:
OtherVersions
OtherVersions
PCUsing
453/479
29/7/2016
TheGNUAwkUsersGuide
PCoperatingsystems,gawkon,installing:
percentsign(%),%operator:
percentsign(%),%=operator:
percentsign(%),%=operator:
period(.),regexpoperator:
Perl:
Peters,Arno:
Peterson,Hal:
pipe,closing:
pipe,input:
pipe,output:
Pitts,Dave:
Pitts,Dave:
Plauger,P.J.:
plugin:
plussign(+),+operator:
plussign(+),+operator:
plussign(+),++operator:
plussign(+),+=operator:
plussign(+),+=operator:
plussign(+),regexpoperator:
pointerstofunctions:
portability:
portability,#!(executablescripts):
portability,**operatorand:
portability,**=operatorand:
portability,ARGVvariable:
portability,backslashcontinuationand:
portability,backslashinescapesequences:
portability,close()functionand:
portability,datafilesassinglerecord:
portability,deletingarrayelements:
portability,exampleprograms:
portability,functions,defining:
portability,gawk:
portability,gettextlibraryand:
portability,internationalizationand:
portability,length()function:
portability,newawkvs.oldawk:
portability,nextstatementinuserdefinedfunctions:
portability,NFvariable,decrementing:
PCInstallation
Precedence
AssignmentOps
Precedence
RegexpOperators
FutureExtensions
Contributors
Contributors
CloseFilesAndPipes
Getline/Pipe
Redirection
Acknowledgments
Bugs
LibraryFunctions
ExtensionIntro
Precedence
Precedence
IncrementOps
IncrementOps
Precedence
AssignmentOps
Precedence
RegexpOperators
IndirectCalls
EscapeSequences
ExecutableScripts
ArithmeticOps
AssignmentOps
ExecutableScripts
Statements/Lines
EscapeSequences
CloseFilesAndPipes
gawksplitrecords
Delete
LibraryFunctions
DefinitionSyntax
NewPorts
Explaininggettext
I18NPortability
StringFunctions
StringsAndNumbers
ChangingFields
portability,operators:
portability,operators,notinPOSIXawk:
portability,POSIXLY_CORRECTenvironmentvariable:
portability,substr()function:
IncrementOps
Precedence
Options
StringFunctions
454/479
29/7/2016
TheGNUAwkUsersGuide
portableobjectfiles:
portableobjectfiles:
portableobjectfiles,convertingtomessageobjectfiles:
portableobjectfiles,generating:
portableobjecttemplatefiles:
portinggawk:
positionalspecifiers,printfstatement:
positionalspecifiers,printfstatement:
positionalspecifiers,printfstatement,mixingwithregularformats:
POSIXawk:
POSIXawk:
POSIXawk,**operatorand:
POSIXawk,**=operatorand:
POSIXawk,<operatorand:
POSIXawk,arithmeticoperatorsand:
POSIXawk,backslashesinstringconstants:
POSIXawk,BEGIN/ENDpatterns:
POSIXawk,bracketexpressionsand:
POSIXawk,bracketexpressionsand,characterclasses:
POSIXawk,bracketexpressionsand,characterclasses:
POSIXawk,breakstatementand:
POSIXawk,changesinawkversions:
POSIXawk,continuestatementand:
POSIXawk,CONVFMTvariableand:
POSIXawk,dateutilityand:
POSIXawk,fieldseparatorsand:
POSIXawk,fieldseparatorsand:
POSIXawk,FSvariableand:
POSIXawk,functionkeywordin:
POSIXawk,functionsand,gsub()/sub():
POSIXawk,functionsand,length():
POSIXawk,GNUlongoptionsand:
POSIXawk,intervalexpressionsin:
POSIXawk,next/nextfilestatementsand:
POSIXawk,numericstringsand:
POSIXawk,OFMTvariableand:
POSIXawk,OFMTvariableand:
POSIXawk,period(.),using:
POSIXawk,printfformatstringsand:
POSIXawk,regularexpressionsand:
POSIXawk,timestampsand:
POSIXawk,|I/Ooperatorand:
Explaininggettext
Translatori18n
I18NExample
Options
Explaininggettext
NewPorts
FormatModifiers
PrintfOrdering
PrintfOrdering
ThisManual
AssignmentOps
Precedence
AssignmentOps
Getline/File
ArithmeticOps
EscapeSequences
I/OAndBEGIN/END
BracketExpressions
BracketExpressions
BracketExpressions
BreakStatement
POSIX
ContinueStatement
Usermodified
TimeFunctions
Fields
FullLineFields
Usermodified
DefinitionSyntax
GoryDetails
StringFunctions
Options
RegexpOperators
NextStatement
VariableTyping
OFMT
StringsAndNumbers
RegexpOperators
FormatModifiers
RegexpOperators
TimeFunctions
Getline/Pipe
POSIXmode:
POSIXmode:
POSIX,awkand:
POSIX,gawkextensionsnotincludedin:
Options
Options
Preface
POSIX/GNU
455/479
29/7/2016
TheGNUAwkUsersGuide
POSIX,programs,implementinginawk:
POSIXLY_CORRECTenvironmentvariable:
PRECvariable:
precedence:
precedence:
precedence,regexpoperators:
predefinedvariables:
predefinedvariables,voption,settingwith:
predefinedvariables,conveyinginformation:
predefinedvariables,usermodifiable:
printdebuggercommand:
printstatement:
printstatement,BEGIN/ENDpatternsand:
printstatement,commas,omitting:
printstatement,I/Ooperatorsin:
printstatement,linecontinuationsand:
printstatement,OFMTvariableand:
printstatement,SeeAlsoredirection,ofoutput:
printstatement,sprintf()functionand:
printvariables,indebugger:
printfdebuggercommand:
printfstatement:
printfstatement:
printfstatement,columns,aligning:
printfstatement,formatcontrolcharacters:
printfstatement,I/Ooperatorsin:
printfstatement,modifiers:
printfstatement,positionalspecifiers:
printfstatement,positionalspecifiers:
printfstatement,positionalspecifiers,mixingwithregularformats:
printfstatement,SeeAlsoredirection,ofoutput:
printfstatement,sprintf()functionand:
printfstatement,syntaxof:
printing:
printingmessagesfromextensions:
printing,listofoptions:
printing,mailinglabels:
printing,unduplicatedlinesoftext:
printing,userinformation:
privatevariables:
processgroupIDofgawkprocess:
processIDofgawkprocess:
processes,twowaycommunicationswith:
Clones
Options
Usermodified
IncrementOps
Precedence
RegexpOperators
BuiltinVariables
Options
Autoset
Usermodified
Printing
I/OAndBEGIN/END
PrintExamples
Precedence
PrintExamples
Usermodified
Redirection
RoundFunction
Printing
Printf
PrintExamples
ControlLetters
Precedence
FormatModifiers
FormatModifiers
PrintfOrdering
PrintfOrdering
Redirection
RoundFunction
BasicPrintf
Printing
PrintingMessages
Options
LabelsProgram
UniqProgram
IdProgram
LibraryNames
Autoset
Autoset
TwowayI/O
processingdata:
PROCINFOarray:
PROCINFOarray:
PROCINFOarray:
BasicHighLevel
Autoset
TimeFunctions
PasswdFunctions
456/479
29/7/2016
TheGNUAwkUsersGuide
PROCINFOarray,andcommunicationsviaptys:
PROCINFOarray,andgroupmembership:
PROCINFOarray,anduserandgroupIDnumbers:
PROCINFOarray,testingthefieldsplitting:
PROCINFOarray,uses:
PROCINFO,valuesofsorted_in:
profilingawkprograms:
profilingawkprograms,dynamically:
programidentifiers:
program,definitionof:
programmingconventions,nondecimaldataoption:
programmingconventions,ARGC/ARGVvariables:
programmingconventions,exitstatement:
programmingconventions,functionparameters:
programmingconventions,functions,calling:
programmingconventions,functions,writing:
programmingconventions,gawkextensions:
programmingconventions,privatevariablenames:
programminglanguage,recipefor:
programminglanguages,Ada:
programminglanguages,datadrivenvs.procedural:
programminglanguages,Java:
programming,basicsteps:
programming,concepts:
programming,concepts:
pwcatprogram:
TwowayI/O
GroupFunctions
IdProgram
PasswdFunctions
Autoset
ControllingScanning
Profiling
Profiling
Autoset
GettingStarted
NondecimalData
Autoset
ExitStatement
ReturnStatement
CallingBuiltin
DefinitionSyntax
InternalFileOps
LibraryNames
History
Glossary
GettingStarted
Glossary
BasicHighLevel
BasicConcepts
BasicConcepts
PasswdFunctions
Q
qdebuggercommand(aliasforquit):
QSEawk:
Quanstrom,Erik:
questionmark(?),?:operator:
questionmark(?),regexpoperator:
questionmark(?),regexpoperator:
QuikTrimAwk:
quitdebuggercommand:
QUITsignal(MSWindows):
quotingingawkcommandlines:
quotingingawkcommandlines,tricksfor:
quoting,forsmallawkprograms:
OtherVersions
AlarmProgram
Precedence
RegexpOperators
GNURegexpOperators
OtherVersions
Profiling
Long
Quoting
Comments
R
rdebuggercommand(aliasforrun):
Rakitzis,Byron:
Ramey,Chet:
Ramey,Chet:
rand()function:
HistorySorting
Acknowledgments
GeneralDataTypes
NumericFunctions
457/479
29/7/2016
TheGNUAwkUsersGuide
randomnumbers,Cliff:
randomnumbers,rand()/srand()functions:
randomnumbers,seedof:
rangeexpressions(regexps):
rangepatterns:
rangepatterns,linecontinuationand:
Rankin,Pat:
Rankin,Pat:
Rankin,Pat:
reada()extensionfunction:
readabledatafiles,checking:
readable.awkprogram:
readdirextension:
readfile()extensionfunction:
readfile()userdefinedfunction:
readinginputfiles:
recipeforaprogramminglanguage:
recordseparators:
recordseparators:
recordseparators,changing:
recordseparators,regularexpressionsas:
recordseparators,withmultilinerecords:
records:
records:
records,multiline:
records,printing:
records,splittinginputinto:
records,terminating:
records,treatingfilesas:
recursivefunctions:
redirectgawkoutput,indebugger:
redirectionofinput:
redirectionofoutput:
referencecounting,sortingarrays:
regexp:
regexpconstants:
regexpconstants:
regexpconstants:
regexpconstants,/=/,/=operatorand:
regexpconstants,aspatterns:
regexpconstants,ingawk:
regexpconstants,slashesvs.quotes:
CliffRandomFunction
NumericFunctions
NumericFunctions
BracketExpressions
Ranges
Ranges
Acknowledgments
AssignmentOps
Contributors
ExtensionSampleReadwritearray
FileChecking
FileChecking
ExtensionSampleReaddir
ExtensionSampleReadfile
ReadfileFunction
ReadingFiles
History
awksplitrecords
Usermodified
awksplitrecords
awksplitrecords
MultipleLine
ReadingFiles
BasicHighLevel
MultipleLine
Print
Records
awksplitrecords
gawksplitrecords
DefinitionSyntax
DebuggerInfo
Getline/File
Redirection
Regexp
RegexpUsage
RegexpConstants
ComparisonOperators
AssignmentOps
ExpressionPatterns
ComputedRegexps
regexpconstants,vs.stringconstants:
registerextension:
regularexpressions:
regularexpressionsasfieldseparators:
ComputedRegexps
RegistrationFunctions
Regexp
FieldSeparators
458/479
29/7/2016
TheGNUAwkUsersGuide
regularexpressions,anchorsin:
regularexpressions,asfieldseparators:
regularexpressions,aspatterns:
regularexpressions,aspatterns:
regularexpressions,asrecordseparators:
regularexpressions,casesensitivity:
regularexpressions,casesensitivity:
regularexpressions,computed:
regularexpressions,constants,Seeregexpconstants:
regularexpressions,dynamic:
regularexpressions,dynamic,withembeddednewlines:
regularexpressions,gawk,commandlineoptions:
regularexpressions,intervalexpressionsand:
regularexpressions,leftmostlongestmatch:
regularexpressions,operators:
regularexpressions,operators:
regularexpressions,operators,forbuffers:
regularexpressions,operators,forwords:
regularexpressions,operators,gawk:
regularexpressions,operators,precedenceof:
regularexpressions,searchingfor:
relationaloperators,Seecomparisonoperators:
replaceinstring:
returndebuggercommand:
returnstatement,userdefinedfunctions:
returnvalue,close()function:
rev()userdefinedfunction:
revoutputextension:
revtwowayextension:
rewind()userdefinedfunction:
rightanglebracket(>),>operator:
rightanglebracket(>),>operator:
rightanglebracket(>),>operator(I/O):
rightanglebracket(>),>=operator:
rightanglebracket(>),>=operator:
rightanglebracket(>),>>operator(I/O):
rightanglebracket(>),>>operator(I/O):
rightshift:
rightshift,bitwise:
Ritchie,Dennis:
RLENGTHvariable:
RLENGTHvariable,match()functionand:
Robbins,Arnold:
RegexpOperators
RegexpUsage
RegexpPatterns
awksplitrecords
Casesensitivity
Usermodified
ComputedRegexps
RegexpUsage
ComputedRegexps
ComputedRegexps
GNURegexpOperators
Options
LeftmostLongest
RegexpUsage
RegexpOperators
GNURegexpOperators
GNURegexpOperators
GNURegexpOperators
RegexpOperators
EgrepProgram
TypingandComparison
StringFunctions
ReturnStatement
CloseFilesAndPipes
FunctionExample
ExtensionSampleRevout
ExtensionSampleRev2way
RewindFunction
ComparisonOperators
Precedence
Redirection
ComparisonOperators
Precedence
Redirection
Precedence
BitwiseFunctions
BitwiseFunctions
BasicDataTyping
Autoset
StringFunctions
Robbins,Arnold:
Robbins,Arnold:
Robbins,Arnold:
Robbins,Arnold:
Getline/Pipe
PasswdFunctions
AlarmProgram
GeneralDataTypes
459/479
29/7/2016
TheGNUAwkUsersGuide
Robbins,Arnold:
Robbins,Arnold:
Robbins,Arnold:
Robbins,Bill:
Robbins,Harry:
Robbins,Jean:
Robbins,Miriam:
Robbins,Miriam:
Robbins,Miriam:
Rommel,KaiUwe:
roundtonearestinteger:
round()userdefinedfunction:
roundingnumbers:
ROUNDMODEvariable:
RSvariable:
RSvariable:
RSvariable,multilinerecordsand:
rshift()function(gawk):
RSTARTvariable:
RSTARTvariable,match()functionand:
RTvariable:
RTvariable:
RTvariable:
Rubin,Paul:
Rubin,Paul:
rule,definitionof:
rundebuggercommand:
rvalues/lvalues:
Contributors
Bugs
FutureExtensions
Getline/Pipe
Acknowledgments
Acknowledgments
Acknowledgments
Getline/Pipe
PasswdFunctions
Contributors
NumericFunctions
RoundFunction
RoundFunction
Usermodified
awksplitrecords
Usermodified
MultipleLine
BitwiseFunctions
Autoset
StringFunctions
awksplitrecords
MultipleLine
Autoset
History
Contributors
GettingStarted
AssignmentOps
sdebuggercommand(aliasforstep):
sampledebuggingsession:
sandboxmode:
savedebuggeroptions:
scalarorarray:
scalarvalues:
scanningarrays:
scanningmultidimensionalarrays:
Schorr,Andrew:
Schorr,Andrew:
Schorr,Andrew:
Schreiber,Bert:
SampleDebuggingSession
Options
DebuggerInfo
TypeFunctions
BasicDataTyping
ScanninganArray
Multiscanning
Acknowledgments
Autoset
Contributors
Acknowledgments
Schreiber,Rita:
searchandreplaceinstrings:
searchinstring:
searchpaths:
searchpaths:
Acknowledgments
StringFunctions
StringFunctions
ProgramsExercises
PCUsing
460/479
29/7/2016
TheGNUAwkUsersGuide
searchpaths:
searchpaths,forloadableextensions:
searchpaths,forsourcefiles:
searching,filesforregularexpressions:
searching,forwords:
sedutility:
sedutility:
sedutility:
seedingrandomnumbergenerator:
semicolon(;),AWKPATHvariableand:
semicolon(;),separatingstatementsinactions:
separators,field:
separators,field:
separators,field,FIELDWIDTHSvariableand:
separators,field,FPATvariableand:
separators,field,POSIXand:
separators,forrecords:
separators,forrecords,regularexpressionsas:
separators,forstatementsinactions:
separators,subscript:
setbreakpoint:
setdebuggercommand:
setdirectoryofmessagecatalogs:
setwatchpoint:
shadowingofvariablevalues:
shellquoting,rulesfor:
shells,pipingcommandsinto:
shells,quoting:
shells,quoting,rulesfor:
shells,scripts:
shells,sea:
shells,variables:
shift,bitwise:
shortcircuitoperators:
showallsourcefiles,indebugger:
VMSRunning
AWKLIBPATHVariable
AWKPATHVariable
ProgramsExercises
PCUsing
VMSRunning
EgrepProgram
DupwordProgram
FullLineFields
SimpleSed
Glossary
NumericFunctions
PCUsing
Statements/Lines
ActionOverview
Statements
Usermodified
Usermodified
Usermodified
Usermodified
Fields
awksplitrecords
awksplitrecords
Usermodified
awksplitrecords
ActionOverview
Usermodified
BreakpointControl
I18NFunctions
DefinitionSyntax
Quoting
Redirection
UsingShellVariables
Quoting
Oneshot
Undocumented
UsingShellVariables
BitwiseFunctions
BooleanOps
DebuggerInfo
showbreakpoints:
showfunctionarguments,indebugger:
showlocalvariables,indebugger:
shownameofcurrentsourcefile,indebugger:
showwatchpoints:
DebuggerInfo
DebuggerInfo
DebuggerInfo
DebuggerInfo
DebuggerInfo
461/479
29/7/2016
TheGNUAwkUsersGuide
sidebuggercommand(aliasforstepi):
sideeffects:
Concatenation
sideeffects:
IncrementOps
sideeffects:
IncrementOps
sideeffects,arrayindexing:
ReferencetoElements
sideeffects,asort()function:
sideeffects,assignmentexpressions:
AssignmentOps
sideeffects,Booleanoperators:
BooleanOps
sideeffects,conditionalexpressions:
ConditionalExp
sideeffects,decrement/incrementoperators:
IncrementOps
sideeffects,FILENAMEvariable:
GetlineNotes
sideeffects,functioncalls:
FunctionCalls
sideeffects,statements:
ActionOverview
sidebar,AConstantsBaseDoesNotAffectItsValue:
Nondecimalnumbers
sidebar,BackslashBeforeRegularCharacters:
EscapeSequences
sidebar,ChangingFSDoesNotAffecttheFields:
FullLineFields
sidebar,ChangingNRandFNR:
Autoset
sidebar,ControllingOutputBufferingwithsystem():
I/OFunctions
sidebar,EscapeSequencesforMetacharacters:
EscapeSequences
sidebar,FSandIGNORECASE:
FieldSplittingSummary
sidebar,InteractiveVersusNoninteractiveBuffering:
I/OFunctions
sidebar,MatchingtheNullString:
StringFunctions
sidebar,OperatorEvaluationOrder:
IncrementOps
sidebar,Pipingintosh:
Redirection
sidebar,PrePOSIXawkUsedOFMTforStringConversion:
StringsAndNumbers
sidebar,RecipeforaProgrammingLanguage:
History
sidebar,RS="\0"IsNotPortable:
gawksplitrecords
sidebar,SoWhyDoesgawkHaveBEGINFILEandENDFILE?:
FiletransFunction
sidebar,SyntacticAmbiguitiesBetween/=andRegularExpressions: AssignmentOps
sidebar,Understanding#!:
ExecutableScripts
sidebar,Understanding$0:
ChangingFields
sidebar,Usingclose()sReturnValue:
CloseFilesAndPipes
sidebar,Using\ninBracketExpressionsofDynamicRegexps:
ComputedRegexps
SIGHUPsignal,fordynamicprofiling:
Profiling
SIGINTsignal(MSWindows):
Profiling
signals,HUP/SIGHUP,forprofiling:
Profiling
signals,INT/SIGINT(MSWindows):
Profiling
signals,QUIT/SIGQUIT(MSWindows):
Profiling
signals,USR1/SIGUSR1,forprofiling:
Profiling
signatureprogram:
SignatureProgram
SIGQUITsignal(MSWindows):
Profiling
SIGUSR1signal,fordynamicprofiling:
Profiling
silentdebuggercommand:
sin()function:
NumericFunctions
sine:
NumericFunctions
singlequote('):
Oneshot
singlequote(')ingawkcommandlines:
Long
462/479
29/7/2016
singlequote('),inshellcommands:
singlequote('),vs.apostrophe:
singlequote('),withdoublequotes:
singlecharacterfields:
singlestepexecution,inthedebugger:
Skywalker,Luke:
sleeputility:
sleep()extensionfunction:
Solaris,POSIXcompliantawk:
sortarray:
sortarrayindices:
sortfunction,arrays,sorting:
sortutility:
sortutility,coprocessesand:
sortingcharactersindifferentlanguages:
sourcecode,awka:
sourcecode,BrianKernighansawk:
sourcecode,BusyBoxAwk:
sourcecode,gawk:
sourcecode,Illumosawk:
sourcecode,jawk:
sourcecode,libmawk:
sourcecode,mawk:
sourcecode,mixing:
sourcecode,pawk:
sourcecode,pawk(Pythonversion):
sourcecode,QSEawk:
sourcecode,QuikTrimAwk:
sourcecode,Solarisawk:
sourcefiles,searchpathfor:
sparsearrays:
Spencer,Henry:
splitstringintoarray:
splitutility:
split()function:
split()function,arrayelements,deleting:
split.awkprogram:
sprintf()function:
sprintf()function:
sprintf()function,OFMTvariableand:
sprintf()function,print/printfstatementsand:
sqrt()function:
squarebrackets([]),regexpoperator:
squareroot:
srand()function:
stackframe:
Stallman,Richard:
TheGNUAwkUsersGuide
Quoting
Comments
Quoting
Undocumented
AlarmProgram
ExtensionSampleTime
OtherVersions
StringFunctions
StringFunctions
WordSorting
TwowayI/O
Explaininggettext
OtherVersions
OtherVersions
OtherVersions
GawkDistribution
OtherVersions
OtherVersions
OtherVersions
OtherVersions
Options
OtherVersions
OtherVersions
OtherVersions
OtherVersions
OtherVersions
ProgramsExercises
ArrayIntro
Glossary
StringFunctions
SplitProgram
StringFunctions
Delete
SplitProgram
OFMT
StringFunctions
Usermodified
RoundFunction
NumericFunctions
RegexpOperators
NumericFunctions
NumericFunctions
DebuggingTerms
ManualHistory
463/479
29/7/2016
TheGNUAwkUsersGuide
Stallman,Richard:
Stallman,Richard:
Stallman,Richard:
standarderror:
standardinput:
standardinput:
standardoutput:
startingthedebugger:
stat()extensionfunction:
statements,compound,controlstatementsand:
statements,control,inactions:
statements,multiple:
stepdebuggercommand:
stepidebuggercommand:
stopautomaticdisplay,indebugger:
streameditors:
streameditors:
strftime()function(gawk):
stringconstants:
stringconstants,vs.regexpconstants:
stringextraction(internationalization):
stringlength:
stringoperators:
string,regularexpressionmatch:
stringmanipulationfunctions:
stringmatchingoperators:
stringtranslationfunctions:
stringssplitting,example:
strings,converting:
strings,converting:
strings,convertinglettercase:
strings,converting,numbersto:
strings,converting,numbersto:
strings,empty,Seenullstrings:
strings,extracting:
strings,forlocalization:
strings,lengthlimitations:
strings,mergingarraysinto:
strings,null:
strings,numeric:
Acknowledgments
Contributors
Glossary
SpecialFD
ReadTerminal
SpecialFD
SpecialFD
DebuggerInvocation
Statements
Statements
Statements/Lines
FullLineFields
SimpleSed
TimeFunctions
ScalarConstants
ComputedRegexps
StringExtraction
StringFunctions
Concatenation
StringFunctions
StringFunctions
RegexpUsage
I18NFunctions
StringFunctions
StringsAndNumbers
BitwiseFunctions
StringFunctions
Usermodified
Usermodified
awksplitrecords
StringExtraction
Programmeri18n
ScalarConstants
JoinFunction
VariableTyping
strtonum()function(gawk):
StringFunctions
NondecimalData
StringFunctions
StringFunctions
GoryDetails
Usermodified
strtonum()function(gawk),nondecimaldataoptionand:
sub()function:
sub()function:
sub()function,argumentsof:
sub()function,escapeprocessing:
subscriptseparators:
464/479
29/7/2016
TheGNUAwkUsersGuide
subscriptsinarrays,multidimensional:
subscriptsinarrays,multidimensional,scanning:
subscriptsinarrays,numbersas:
subscriptsinarrays,uninitializedvariablesas:
SUBSEPvariable:
SUBSEPvariable,andmultidimensionalarrays:
substituteinstring:
substr()function:
substring:
Sumner,Andrew:
supplementarygroupsofgawkprocess:
switchstatement:
SYMTABarray:
syntacticambiguity:/=operatorvs./=/regexpconstant:
system()function:
systime()function(gawk):
Multidimensional
Multiscanning
Usermodified
Multidimensional
StringFunctions
StringFunctions
StringFunctions
OtherVersions
Autoset
SwitchStatement
Autoset
AssignmentOps
I/OFunctions
TimeFunctions
tdebuggercommand(aliasfortbreak):
BreakpointControl
BreakpointControl
LibraryNames
TCP/IPNetworking
SpecialNetwork
TeeProgram
TeeProgram
BreakpointControl
awksplitrecords
BitwiseFunctions
ExtensionSampleAPITests
Conventions
LibraryFunctions
DupwordProgram
ExtractProgram
AddingCode
RegexpOperators
ExtractProgram
Print
UniqProgram
Usermodified
Programmeri18n
Programmeri18n
I18NPortability
Explaininggettext
RegexpUsage
ComputedRegexps
Casesensitivity
T
tbreakdebuggercommand:
Tcl:
TCP/IP:
TCP/IP,supportfor:
teeutility:
tee.awkprogram:
temporarybreakpoint:
terminatingrecords:
testbits.awkprogram:
testextextension:
Texinfo:
Texinfo:
Texinfo:
Texinfo:
Texinfo:
Texinfo:
Texinfo,chapterbeginningsinfiles:
Texinfo,extractingprogramsfromsourcefiles:
text,printing:
text,printing,unduplicatedlinesof:
TEXTDOMAINvariable:
TEXTDOMAINvariable:
TEXTDOMAINvariable,BEGINpatternand:
TEXTDOMAINvariable,portabilityand:
textdomain()function(Clibrary):
tilde(~),~operator:
tilde(~),~operator:
tilde(~),~operator:
465/479
29/7/2016
TheGNUAwkUsersGuide
tilde(~),~operator:
tilde(~),~operator:
tilde(~),~operator:
tilde(~),~operator:
tilde(~),~operator:
timefunctions:
time,alarmclockexampleprogram:
time,localizationand:
time,managing:
time,retrieving:
timeout,readinginput:
timestamps:
timestamps:
timestamps,convertingdatesto:
timestamps,formatted:
tolower()function:
toupper()function:
trutility:
tracedebuggercommand:
traceback,displayindebugger:
translatestring:
translate.awkprogram:
treatingfiles,assinglerecords:
troubleshooting,nondecimaldataoption:
troubleshooting,==operator:
troubleshooting,awkusesFSnotIFS:
troubleshooting,backslashbeforenonspecialcharacter:
troubleshooting,division:
troubleshooting,fatalerrors,fieldwidths,specifying:
troubleshooting,fatalerrors,printfformatstrings:
troubleshooting,fflush()function:
troubleshooting,functioncallsyntax:
troubleshooting,gawk:
troubleshooting,gawk,bugreports:
troubleshooting,gawk,fatalerrors,functionarguments:
troubleshooting,getlinefunction:
troubleshooting,gsub()/sub()functions:
troubleshooting,match()function:
troubleshooting,printstatement,omittingcommas:
RegexpConstants
ComparisonOperators
ComparisonOperators
Precedence
ExpressionPatterns
TimeFunctions
AlarmProgram
Explaininggettext
TimeFunctions
ReadTimeout
TimeFunctions
TimeFunctions
TimeFunctions
StringFunctions
StringFunctions
TranslateProgram
ExecutionStack
I18NFunctions
TranslateProgram
gawksplitrecords
Options
ComparisonOperators
FieldSeparators
EscapeSequences
ArithmeticOps
ConstantSize
FormatModifiers
I/OFunctions
FunctionCalls
CompatibilityMode
Bugs
CallingBuiltin
FileChecking
StringFunctions
StringFunctions
PrintExamples
troubleshooting,printing:
troubleshooting,quoteswithfilenames:
troubleshooting,readabledatafiles:
troubleshooting,regexpconstantsvs.stringconstants:
troubleshooting,stringconcatenation:
troubleshooting,substr()function:
troubleshooting,system()function:
troubleshooting,typographicalerrors,globalvariables:
Redirection
SpecialFD
FileChecking
ComputedRegexps
Concatenation
StringFunctions
I/OFunctions
Options
466/479
29/7/2016
TheGNUAwkUsersGuide
true,logical:
Trueman,David:
Trueman,David:
Trueman,David:
truncmodoperation:
truthvalues:
typeconversion:
TruthValues
History
Acknowledgments
Contributors
ArithmeticOps
TruthValues
StringsAndNumbers
udebuggercommand(aliasforuntil):
unassignedarrayelements:
undefinedfunctions:
underscore(_),Cmacro:
underscore(_),innamesofprivatevariables:
underscore(_),translatablestring:
undisplaydebuggercommand:
undocumentedfeatures:
Unicode:
Unicode:
Unicode:
uninitializedvariables,asarraysubscripts:
uniqutility:
uniq.awkprogram:
Unix:
Unixawk,backslashesinescapesequences:
Unixawk,close()functionand:
Unixawk,passwordfiles,fieldseparatorsand:
Unix,awkscriptsand:
UNIXROOTvariable,onOS/2systems:
unsignedintegers:
untildebuggercommand:
unwatchdebuggercommand:
updebuggercommand:
userdatabase,reading:
userdefinedfunctions:
userdefined,functions,counts,inaprofile:
userdefined,variables:
usermodifiablevariables:
users,informationabout,printing:
ReferencetoElements
Explaininggettext
LibraryNames
Programmeri18n
Undocumented
OrdinalFunctions
RangesandLocales
Glossary
UniqProgram
UniqProgram
Glossary
EscapeSequences
CloseFilesAndPipes
ExecutableScripts
PCUsing
ComputerArithmetic
ExecutionStack
PasswdFunctions
Userdefined
Profiling
Variables
Usermodified
IdProgram
users,informationabout,retrieving:
USR1signal,fordynamicprofiling:
PasswdFunctions
Profiling
values,numeric:
values,string:
variableassignmentsandinputfiles:
BasicDataTyping
BasicDataTyping
OtherArguments
467/479
29/7/2016
TheGNUAwkUsersGuide
variabletyping:
variables:
variables:
variables,assigningoncommandline:
variables,builtin:
variables,flag:
variables,getlinecommandinto,using:
variables,global,forlibraryfunctions:
variables,global,printinglistof:
variables,initializing:
variables,localtoafunction:
variables,predefined:
variables,predefinedvoption,settingwith:
variables,predefinedconveyinginformation:
variables,private:
variables,setting:
variables,shadowing:
variables,typesof:
variables,typesof,comparisonexpressionsand:
variables,uninitialized,asarraysubscripts:
variables,userdefined:
versionofgawk:
versionofgawkextensionAPI:
versionofGNUMPlibrary:
versionofGNUMPFRlibrary:
verticalbar(|):
verticalbar(|),|operator(I/O):
verticalbar(|),|operator(I/O):
verticalbar(|),|&operator(I/O):
verticalbar(|),||operator:
verticalbar(|),||operator:
Vinschen,Corinna:
TypingandComparison
OtherFeatures
BasicDataTyping
AssignmentOptions
UsingVariables
BooleanOps
Getline/Variable
Getline/Variable/File
Getline/Variable/Pipe
Getline/Variable/Coprocess
LibraryNames
Options
UsingVariables
VariableScope
BuiltinVariables
Options
Autoset
LibraryNames
Options
DefinitionSyntax
AssignmentOps
TypingandComparison
Variables
Autoset
Autoset
Autoset
Autoset
RegexpOperators
Getline/Pipe
Precedence
Getline/Coprocess
Precedence
TwowayI/O
BooleanOps
Precedence
Acknowledgments
wdebuggercommand(aliasforwatch):
wutility:
ConstantSize
ExtensionSampleFork
ExtensionSampleFork
WalkingArrays
ArrayIntro
FutureExtensions
Contributors
wait()extensionfunction:
waitpid()extensionfunction:
walk_array()userdefinedfunction:
Wall,Larry:
Wall,Larry:
Wallin,Anders:
468/479
29/7/2016
TheGNUAwkUsersGuide
warnings,issuing:
watchdebuggercommand:
watchpoint:
wcutility:
wc.awkprogram:
Weinberger,Peter:
Weinberger,Peter:
wheredebuggercommand:
wheredebuggercommand(aliasforbacktrace):
whilestatement:
whilestatement,useofregexpsin:
whitespace,asfieldseparators:
whitespace,functions,calling:
whitespace,newlinesas:
Williams,Kent:
Woehlke,Matthew:
Woods,John:
wordboundaries,matching:
word,regexpdefinitionof:
wordboundaryoperator(gawk):
wordfreq.awkprogram:
words,counting:
words,duplicate,searchingfor:
words,usagecounts,generating:
writea()extensionfunction:
Options
DebuggingTerms
WcProgram
WcProgram
History
Contributors
ExecutionStack
ExecutionStack
WhileStatement
RegexpUsage
DefaultFieldSplitting
CallingBuiltin
Options
Contributors
Contributors
Contributors
GNURegexpOperators
GNURegexpOperators
GNURegexpOperators
WordSorting
WcProgram
DupwordProgram
WordSorting
ExtensionSampleReadwritearray
xgettextutility:
XORbitwiseoperation:
xor()function(gawk):
StringExtraction
BitwiseFunctions
BitwiseFunctions
Yawitz,Efraim:
Contributors
Zaretskii,Eli:
Zaretskii,Eli:
Zaretskii,Eli:
zerofile.awkprogram:
Zoulas,Christos:
Acknowledgments
Contributors
Bugs
EmptyFiles
Contributors
Jumpto: !"#$%&'(*+,./<=>?@[\^_{|~
ABCDEFGHIJKLMNOPQRSTUVWXYZ
Footnotes
469/479
29/7/2016
TheGNUAwkUsersGuide
(1)
The2008POSIXstandardisaccessibleonlineathttp://www.opengroup.org/onlinepubs/9699919799/.
(2)
TheseutilitiesareavailableonPOSIXcompliantsystems,aswellasontraditionalUnixbasedsystems.Ifyouareusing
someotheroperatingsystem,youstillneedtobefamiliarwiththeideasofI/Oredirectionandpipes.
(3)
Someother,obsoletesystemstowhichgawkwasonceportedarenolongersupportedandthecodeforthosesystemshas
beenremoved.
(4)
OnlySolarissystemsstilluseanoldawkforthedefaultawkutility.Amoremodernawklivesin/usr/xpg6/binonthese
systems.
(5)
Allsuchdifferencesappearintheindexundertheentrydifferencesinawkandgawk.
(6)
GNUstandsforGNUsNotUnix.
(7)
TheterminologyGNU/LinuxisexplainedintheGlossary.
(8)
The#!mechanismworksonGNU/Linuxsystems,BSDbasedsystems,andcommercialUnixsystems.
(9)
The?and:referredtohereisthethreeoperandconditionalexpressiondescribedinConditionalExp.Splittinglines
after?and:isaminorgawkextensionifposixisspecified(seeOptions),thenthisextensionisdisabled.
(10)
OtherpopularscriptinglanguagesincludeRubyandPerl.
(11)
Formoredetail,pleaseseeSection4.4ofRFC3875.Alsoseetheexplanatorynotesenttothegawkbugmailinglist.
(12)
Notrecommended.
(13)
470/479
29/7/2016
TheGNUAwkUsersGuide
SemicolonsonMSWindowsandMSDOS.
(14)
Yourversionofgawkmayuseadifferentdirectoryitwilldependuponhowgawkwasbuiltandinstalled.Theactual
directoryisthevalueof$(datadir)generatedwhengawkwasconfigured.Youprobablydontneedtoworryaboutthis,
though.
(15)
Inotherliterature,youmayseeabracketexpressionreferredtoaseitheracharacterset,acharacterclass,ora
characterlist.
(16)
Usetwobackslashesifyoureusingastringconstantwitharegexpoperatororfunction.
(17)
ExperiencedCandC++programmerswillnotethatitispossible,usingsomethinglikeIGNORECASE=1&&/foObAr/{
}and IGNORECASE=0||/foobar/{}.However,thisissomewhatobscureandwedontrecommendit.
(18)
Ifyoudontunderstandthis,dontworryaboutititjustmeansthatgawkdoestherightthing.
(19)
Atleastthatweknowabout.
(20)
InPOSIXawk,newlinesarenotconsideredwhitespaceforseparatingfields.
(21)
Abinaryoperator,suchas*formultiplication,isonethattakestwooperands.Thedistinctionisrequiredbecauseawk
alsohasunary(oneoperand)andternary(threeoperand)operators.
(22)
ThankstoAndrewSchorrforthistip.
(23)
Thesedutilityisastreameditor.ItsbehaviorisalsodefinedbythePOSIXstandard.
(24)
TheCSVformatlackedaformalstandarddefinitionformanyyears.RFC4180standardizesthemostcommon
practices.
(25)
471/479
29/7/2016
TheGNUAwkUsersGuide
WhenFSisthenullstring("")oraregexp,thisspecialfeatureofRSdoesnotapply.Itdoesapplytothedefaultfield
separatorofasinglespace:FS="".
(26)
Thisisnotquitetrue.RTcouldbechangedifRSisaregularexpression.
(27)
Thisassumesthatstandardinputisthekeyboard.
(28)
Thettyin/dev/ttystandsforTeletype,aserialterminal.
(29)
Thetechnicalterminologyisrathermorbid.Thefinishedchildiscalledazombie,andcleaningupafteritisreferred
toasreaping.
(30)
Thisisafull16bitvalueasreturnedbythewait()systemcall.Seethesystemmanualpagesforinformationonhowto
decodethisvalue.
(31)
Theinternalrepresentationofallnumbers,includingintegers,usesdoubleprecisionfloatingpointnumbers.Onmost
modernsystems,theseareinIEEE754standardformat.SeeArbitraryPrecisionArithmetic,formuchmore
information.
(32)
Pathologicalcasescanrequireupto752digits(!),butwedoubtthatyouneedtoworryaboutthis.
(33)
IthappensthatBWKawk,gawk,andmawkallgetitright,butyoushouldnotrelyonthis.
(34)
Technically,stringcomparisonissupposedtobehavethesamewayasifthestringswerecomparedwiththeC
strcoll()function.
(35)
ThisprogramhasabugitprintslinesstartingwithEND.Howwouldyoufixit?
(36)
Theoriginalversionofawkkeptreadingandignoringinputuntiltheendofthefilewasseen.
(37)
472/479
29/7/2016
TheGNUAwkUsersGuide
InPOSIXawk,newlinedoesnotcountaswhitespace.
(38)
SomeearlyimplementationsofUnixawkinitializedFILENAMEto"",evenifthereweredatafilestobeprocessed.This
behaviorwasincorrectandshouldnotberelieduponinyourprograms.
(39)
Nottomentiondifficultimplementationissues.
(40)
Theorderingwillvaryamongawkimplementations,whichtypicallyusehashtablestostorearrayelementsandvalues.
(41)
Whentwoelementscompareasequal,theCqsort()functiondoesnotguaranteethattheywillmaintaintheiroriginal
relativeorderaftersorting.Usingthestringvaluetoprovideauniqueorderingwhenthenumericvaluesareequal
ensuresthatgawkbehavesconsistentlyacrossdifferentenvironments.
(42)
ThankstoMichaelBrennanforpointingthisout.
(43)
TheCversionofrand()onmanyUnixsystemsisknowntoproducefairlypoorsequencesofrandomnumbers.
However,nothingrequiresthatanawkimplementationusetheCrand()toimplementtheawkversionofrand().Infact,
gawkusestheBSDrandom()function,whichisconsiderablybetterthanrand(),toproducerandomnumbers.
(44)
mawkusesadifferentseedeachtime.
(45)
Computergeneratedrandomnumbersreallyarenottrulyrandom.Theyaretechnicallyknownaspseudorandom.This
meansthatalthoughthenumbersinasequenceappeartoberandom,youcaninfactgeneratethesamesequenceof
randomnumbersoverandoveragain.
(46)
Unlessyouusethenondecimaldataoption,whichisntrecommended.SeeNondecimalData,formoreinformation.
(47)
NotethatthismeansthattherecordwillfirstberegeneratedusingthevalueofOFSifanyfieldshavebeenchanged,and
thatthefieldswillbeupdatedafterthesubstitution,eveniftheoperationisanoopsuchassub(/^/,"").
(48)
ThisisdifferentfromCandC++,inwhichthefirstcharacterisnumberzero.
473/479
29/7/2016
TheGNUAwkUsersGuide
(49)
Thiswasrathernaiveofhim,despitetherebeinganoteinthissectionindicatingthatthenextmajorversionwould
movetothePOSIXrules.
(50)
Aprogramisinteractiveifthestandardoutputisconnectedtoaterminaldevice.Onmodernsystems,thismeansyour
keyboardandscreen.
(51)
SeeGlossary,especiallytheentriesEpochandUTC.
(52)
TheGNUdateutilitycanalsodomanyofthethingsdescribedhere.Itsusemaybepreferableforsimpletimerelated
operationsinshellscripts.
(53)
Occasionallythereareminutesinayearwithaleapsecond,whichiswhythesecondscangoupto60.
(54)
Unfortunately,noteverysystemsstrftime()necessarilysupportsalloftheconversionslistedhere.
(55)
Ifyoudontunderstandanyofthis,dontworryaboutitthesefacilitiesaremeanttomakeiteasiertointernationalize
programs.OtherinternationalizationfeaturesaredescribedinInternationalization.
(56)
ThisisbecauseISOCleavesthebehavioroftheCversionofstrftime()undefinedandgawkusesthesystemsversion
ofstrftime()ifitsthere.Typically,theconversionspecifiereitherdoesnotappearinthereturnedstringorappears
literally.
(57)
Thisexampleshowsthatzeroscomeinontheleftside.Forgawk,thisisalwaystrue,butinsomelanguages,itspossible
tohavetheleftsidefillwithones.
(58)
Thisprogramwontactuallyrun,becausefoo()isundefined.
(59)
Latein2012.
(60)
Thismaychangeinafutureversionrecheckthedocumentationthatcomeswithyourversionofgawktoseeifithas.
474/479
29/7/2016
TheGNUAwkUsersGuide
(61)
Sadly,over35yearslater,manyofthelessonstaughtbythisbookhaveyettobelearnedbyavastnumberofpracticing
programmers.
(62)
Theeffectsarenotidentical.Outputofthetransformedrecordwillbeinalllowercase,whileIGNORECASEpreservesthe
originalcontentsoftheinputrecord.
(63)
Althoughallthelibraryroutinescouldhavebeenrewrittentousethisconvention,thiswasnotdone,inordertoshow
howourownawkprogrammingstylehasevolvedandtoprovidesomebasisforthisdiscussion.
(64)
gawksdumpvariablescommandlineoptionisusefulforverifyingthis.
(65)
ThisischangingmanysystemsuseUnicode,averylargecharactersetthatincludesASCIIasasubset.Onsystems
withfullUnicodesupport,acharactercanoccupyupto32bits,makingsimpletestssuchasusedhereprohibitively
expensive.
(66)
ASCIIhasbeenextendedinmanycountriestousethevaluesfrom128to255forcountryspecificcharacters.Ifyour
systemusestheseextensions,youcansimplify_ord_init()toloopfrom0to255.
(67)
Itwouldbeniceifawkhadanassignmentoperatorforconcatenation.Thelackofanexplicitoperatorforconcatenation
makesstringoperationsmoredifficultthantheyreallyneedtobe.
(68)
TheBEGINFILEspecialpattern(seeBEGINFILE/ENDFILE)providesanalternativemechanismfordealingwithfiles
thatcantbeopened.However,thecodehereprovidesaportablesolution.
(69)
Thisfunctionwaswrittenbeforegawkacquiredtheabilitytosplitstringsintosinglecharactersusing""astheseparator.
Wehaveleftitalone,asusingsubstr()ismoreportable.
(70)
Itisoftenthecasethatpasswordinformationisstoredinanetworkdatabase.
(71)
Thereisasubtleproblemwiththecodejustpresented.Supposethatthefirsttimetherewerenonames.Thiscodeadds
thenameswithaleadingcomma.Italsodoesntcheckthatthereisa$4.
475/479
29/7/2016
TheGNUAwkUsersGuide
(72)
Italsointroducesasubtlebugifamatchhappens,weoutputthetranslatedline,nottheoriginal.
(73)
Thisisthetraditionalusage.ThePOSIXusageisdifferent,butnotrelevantforwhattheprogramaimstodemonstrate.
(74)
Becausegawkunderstandsmultibytelocales,thiscodecountscharacters,notbytes.
(75)
Onsomeoldersystems,includingSolaris,thesystemversionoftrmayrequirethatthelistsbewrittenasrange
expressionsenclosedinsquarebrackets([az])andquoted,topreventtheshellfromattemptingafilename
expansion.Thisisnotafeature.
(76)
Realworldisdefinedasaprogramactuallyusedtogetsomethingdone.
(77)
Fullyexplainingtheshlanguageisbeyondthescopeofthisbook.Weprovidesomeminimalexplanations,butseea
goodshellprogrammingbookifyouwishtounderstandthingsinmoredepth.
(78)
Onsomeveryoldversionsofawk,thetestgetlinejunk<tcanloopforeverifthefileexistsbutisempty.
(79)
gawkdoes@includeprocessingitselfinordertosupporttheuseofawkprogramsasWebCGIscripts.
(80)
Thisisthedefinitionreturnedfromenteringdefine:statemachineintoGoogle.
(81)
Thisiswhythepredefinedsortingordersstartwithan@character,whichcannotbepartofanidentifier.
(82)
ThisistruebecauselocalebasedcomparisonoccursonlywheninPOSIXcompatibilitymode,andbecauseasort()and
asorti()aregawkextensions,theyarenotavailableinthatcase.
(83)
MichaelBrennansuggeststheuseofrand()togenerateuniquefilenames.Thisisavalidpointnevertheless,temporary
filesremainmoredifficulttousethantwowaypipes.
(84)
476/479
29/7/2016
TheGNUAwkUsersGuide
ThisisverydifferentfromthesameoperatorintheCshellandinBash.
(85)
Forsomeoperatingsystems,thegawkportdoesntsupportGNUgettext.Therefore,thesefeaturesarenotavailableif
youareusingoneofthoseoperatingsystems.Sorry.
(86)
Americansuseacommaeverythreedecimalplacesandaperiodforthedecimalpoint,whilemanyEuropeansdo
exactlytheopposite:1,234.56versus1.234,56.
(87)
ThankstoBrunoHaibleforthisexample.
(88)
ThexgettextutilitythatcomeswithGNUgettextcanhandle.awkfiles.
(89)
ThisexampleisborrowedfromtheGNUgettextmanual.
(90)
ThisisgoodfodderforanObfuscatedawkcontest.
(91)
PerhapsitwouldbebetterifitwerecalledHippy.Ah,well.
(92)
Wedontknowwhytheyexpectthis,buttheydo.
(93)
ThankstoMichaelBrennanforthisdescription,whichwehaveparaphrased,andfortheexamples.
(94)
ThereisaverynicepaperonfloatingpointarithmeticbyDavidGoldberg,WhatEveryComputerScientistShould
KnowAboutFloatingPointArithmetic,ACMComputingSurveys23,1(199103):548.Thisisworthreadingifyou
areinterestedinthedetails,butitdoesrequireabackgroundincomputerscience.
(95)
ItispossiblefortheoutputtobecompletelydifferentiftheClibraryinyoursystemdoesnotusetheIEEE754even
roundingruletoroundhalfwaycasesforprintf.
(96)
Weisstein,EricW.SylvestersSequence.FromMathWorldAWolframWebResource
477/479
29/7/2016
TheGNUAwkUsersGuide
(http://mathworld.wolfram.com/SylvestersSequence.html).
(97)
Youaskedforit,yougotit.
(98)
SeethecookieentryintheJargonfileforadefinitionofcookie,andthemagiccookieentryintheJargonfilefora
niceexample.SeealsotheentryforCookieintheGlossary.
(99)
ThisismorecommononMSWindowssystems,butitcanhappenonUnixlikesystemsaswell.
(100)
BecausetheAPIusesonlyISOC90features,itcannotmakeuseoftheISOC99variadicmacrofeaturetohidethat
parameter.Moresthepity.
(101)
Thedifferenceismeasurableandquitereal.Trustus.
(102)
Numericvaluesareclearlylessproblematic,requiringonlyaCdoubletostore.
(103)
OK,theonlydatastructure.
(104)
Itisalsoacookie,butthegawkdevelopersdidnotwishtooverusethisterm.
(105)
Thisversioniseditedslightlyforpresentation.Seeextension/filefuncs.cinthegawkdistributionforthecomplete
version.
(106)
Inpractice,youwouldprobablywanttousetheGNUAutotools(Automake,Autoconf,Libtool,andgettext)to
configureandbuildyourlibraries.InstructionsfordoingsoarebeyondthescopeofthisWebpage.Seegawkextlib,for
Internetlinkstothetools.
(107)
AndLifewasgood.
(108)
AndthuswasborntheCampaignforRationalRangeInterpretation(orRRI).AnumberofGNUtoolshavealready
478/479
29/7/2016
TheGNUAwkUsersGuide
implementedthischange,orwillsoon.ThankstoKarlBerryforcoiningthephraseRationalRangeInterpretation.
(109)
Seethestandardanditsrationale.
(110)
AsofNovember2014,thissiteisstillthere,buttheauthorcouldnotfindapackageforGNUMake.
(111)
TheIA64architectureisalsoknownasItanium.
(112)
Wetried.Itwaspainful.
(113)
ThereisoneGNUprogramthatis(inouropinion)severelydifficulttobootstrapfromtheGitrepository.Forexample,
ontheauthorsold(butstillworking)PowerPCMacintoshwithMacOSX10.5,itwasnecessarytobootstrapatonof
software,startingwithGititself,inordertotrytoworkwiththelatestcode.Itsnotpleasant,andespeciallyonolder
systems,itsabigwasteoftime.
Startingwiththelatesttarballwasnopicniceither.Themaintainershaddropped.gzand.bz2filesandonlydistribute
.tar.xzfiles.Itwasnecessarytobootstrapxzfirst!
(114)
Abranch(sinceremoved)createdbyoneoftheotherdevelopersthatdidnotincludethegeneratedfiles.
(115)
Acriticalcentraldatastructureinsidegawk.
(116)
Thesymbolsarethevariablesandfunctionsdefinedinsidegawk.Accesstothesesymbolsbycodeexternaltogawk
loadeddynamicallyatruntimeisproblematiconMSWindows.
(117)
CompiledprogramsaretypicallywritteninlowerlevellanguagessuchasC,C++,orAda,andthentranslated,or
compiled,intoaformthatthecomputercanexecutedirectly.
479/479

The GNU Awk User's Guide

Enviado por

Dados do documento

Título original

Direitos autorais

Formatos disponíveis

Compartilhar este documento

Compartilhar ou incorporar documento

Opções de compartilhamento

Você considera este documento útil?

Este conteúdo é inapropriado?

Direitos autorais:

Formatos disponíveis

The GNU Awk User's Guide

Enviado por

Direitos autorais:

Formatos disponíveis

29/7/2016

BWKawk mawk gawk Nowstandard

Você também pode gostar