オーストラリア議会の討論のデジタル化、1998 年

node, and used them to split statements wherever one of these patterns was found. After separating the statements, we were able to remove these patterns from the body of text. We also used this method of extracting and later removing unwanted patterns for other pieces of data which did not belong to the debate proceedings, such as sub-debate titles./p> child node, with sub-child nodes called and to differentiate the two. Questions in writing, however, are embedded in their own child node called at the end of the XML file./p> speeches used in all four scripts meant that all questions without notice content was already parsed in order. For the first two scripts, questions and answers were already separated onto their own rows. For the third and fourth scripts, just as we did with the rest of the speech content, we used those patterns of data preceding the text to separate questions and answers. Finally, since questions in writing exist in their own child node we were able to use the same parsing method for all scripts, which was to extract all question and answer elements from the child node./p> nodes to separate speeches. As evident in Fig. 3, nodes are nested within nodes, meaning that the patterns of data from interjection statements were separated out in the process. This meant that we did not need to create lists of names and titles for which to search in the text as we did before. However, we used the same list of general interjection statements on which to separate as was used in the first two scripts. We then did an additional check for statements that may have not been separated due to how they were embedded in the XML, and separated those out where needed. In particular, while most statements were categorized in their own child node and hence captured through pattern-based separation, some were not individually categorized, and had to be split manually in this step./p> nodes contain important data on the MP making each statement. As such, we could extract those data associated with each pattern by parsing one element inward, using the XPath expression “talk.start/talker”. We created a pattern lookup table with these data, and merged it with the main Hansard dataframe by the first pattern detected in each statement. Figure 6 provides an example of that lookup table. This approach enabled us to fill in missing data on each MP speaking using data extracted directly from the XML. Finally, we then used the AustralianPoliticians dataset to fill in other missing data, and flagged for interjections in the same manner as before./p> content in their own nodes that contain the voting data and division result. Since we focus primarily on the spoken Hansard content, our parsing scripts do not necessarily capture all divisions data from House proceedings. Our approach to parsing Hansard in the third and fourth scripts described in the Script Differences section naturally allowed for much of the divisions data to be added to our resulting files for 1998 to March 2011, however the parsing scripts used for May 2011 to September 2022 Hansard did not. To supplement our database and in an effort to fill this divisions data gap, we created an additional file containing all divisions data nested under the XPath “//chamber.xscript//division” from the Hansard files in our time frame. To produce this data file, for each Hansard XML we parsed the , , and child-nodes where they existed, extracted any timestamps where available, and did any additional data cleaning as necessary. We used a series of if-else statements in this script to account for variation in the structure of the node over time. Finally, we then added a date variable to distinguish between sitting days./p> element is the date. Every file passed this test, and we detected one discrepancy in an XML file from 03 June 2009, where its session header contained the wrong date. We validated that our file name and date was correct by checking the official PDF release from that sitting day./p>