Speech is structured with different temporal modulations of energy within different spectral regions.  A recent study by Leong et al. (2017) comparing the temporal modulation structure of infant-directed (IDS) and adult-directed speech (ADS) revealed differences in the power within isolated modulation rates and in the phase synchronization index (PSI) across these two speech registers.  Here, we used the same spectral-amplitude modulation phase hierarchy (S-AMPH) method to analyze and compare the temporal modulation structure across samples of IDS and ADS and across samples of happy and neutral affect ADS.  Given that positive affect is a common IDS feature, the later analysis speaks to the potential role of affect in shaping different modulation patterns. For the IDS vs ADS comparison, we used speech samples (English) developed for a large-scale study of infant preference for IDS (Many Babies Consortium, 2020).  Our analysis did not uncover any significant power differences within any modulation bands, but we observed similar trends to those reported by Leong et al for the theta-beta PSIs in the lower frequency regions.  The short duration of the samples in our analysis likely contribute to this incomplete replication of the Leong et al findings.  For the happy vs neutral affect ADS, we selected English sentence materials from a corpus of adult speakers conveying different affective that were validated with perceptual measures (Pell et al 2009). We observed two findings that align with patterns reported previously by Leong et al. for IDS and ADS. First, in the delta modulation band, happy ADS had significantly higher power than neutral ADS.  Second, for theta-beta PSI scores, neutral ADS had a higher synchronization in lower frequency regions. These parallels suggest that affect contributes to the observed temporal modulation differences between IDS and ADS observed by Leong et al. (2017).