Small Business Administration
Office of Entrepreneurial Development
Impact Study of Entrepreneurial Development Resources
February 2006
Final Report
Order Number: SBAHQ04F0346
Prepared by
1220 19th Street NW
Suite 502
Washington DC 20036
202-223-8877
The statements, findings, conclusions, and recommendations found in this study are those of the
authors and do not necessarily reflect the views of the Office of Entrepreneurial Development, the
United States Small Business Administration, or the United States Government.
Office of Entrepreneurial Development
TABLE OF CONTENTS
TABLE OF CHARTS ............................................................................................................. iii
TABLE OF TABLES ................................................................................................................v
I. INTRODUCTION................................................................................................................ 1
II. EXECUTIVE SUMMARY...................................................................................................3
III. METHODOLOGY........................................................................................................... 10
IV. TOTAL ED RESOURCES COMBINED 2003 VS. 2004 ................................................. 15
V. SBDC 2004 VS. TOTAL ED RESOURCES.......................................................................30
VI. SCORE 2004 VS. TOTAL ED RESOURCES ..................................................................40
VII. WBC 2004 VS. TOTAL ED RESOURCES .....................................................................50
VIII. RESULTS FOLLOW-UP STUDY 2003 CLIENTS ......................................................60
IX. SUMMARY OF QUALITATIVE FINDINGS.................................................................69
X. SUMMARY OF QUANTITATIVE FINDINGS............................................................... 71
XI. FURTHER RESEARCH AND RECOMMENDATIONS .............................................74
XII. APPENDICES.................................................................................................................75
Appendix 1 – Follow-up 2003 Clients Survey Charts............................................................................75
Appendix 2 – Survey Response Values by Research Topic..................................................................78
Appendix 3 – Quantified Survey Results – Initial Survey 2004 ...........................................................82
Appendix 4 – Survey Instrument..............................................................................................................86
Appendix 5 – Recommended Fullow-Up Survey...................................................................................90
Impact Study of Entrepreneurial Development Resources
ii
Office of Entrepreneurial Development
TABLE OF CHARTS
Methodology
Figure 3.1 – Survey Sample Sizes by ED Resource and Segment (FY2004) .................................11
Total ED Resources 2003 vs. 2004
Figure 4.1 – Total ED Resources – Client Segments Served...........................................................16
Figure 4.2 – ED Resources – Revenue and Employment................................................................17
Figure 4.3 – Total ED Resources – Demographic Characteristics .................................................18
Figure 4.4 – Total ED Resources – Age of Business Owner (FY2004).........................................19
Figure 4.5 – Total ED Resources – Primary Business Type (FY2004) ..........................................20
Figure 4.6 – Total ED Resources – How useful was the information you received?...................21
Figure 4.7 – Total ED Resources – Were these services useful in starting or NOT starting a
business?...................................................................................................................................................22
Figure 4.8 – Total ED Resources – Changes in Management Practices ........................................23
Figure 4.9 – Total ED Resources – How have these changes had an impact on your firm?......24
Figure 4.10 – Average Revenues (000s) for Firms in Selected Demographic Categories............25
Figure 4.11 – Average Total Employees for Firms in Selected Demographic Categories...........26
Research Findings - SBDC
Figure 5.1 – SBDC – Distribution of Respondents by Segment (FY2004) ...................................31
Figure 5.2 – SBDC – Demographic Characteristics (FY2004) ........................................................32
Figure 5.3 – SBDC – Age of Business Owner (FY2004) .................................................................33
Figure 5.4 – SBDC – Type of Business (FY2004).............................................................................34
Figure 5.5 – SBDC – How useful was the information you received? (FY2004) .........................35
Figure 5.6 – SBDC – Were these services helpful in starting or NOT starting a business?
(FY2004) ..................................................................................................................................................36
Figure 5.7 – SBDC – As a result of assistance, have you changed management
practices/strategies? (FY2004)..............................................................................................................37
Figure 5.8 – SBDC – Changes in Management Practices (FY2004) ...............................................38
Figure 5.9 – SBDC – How have these changes had an impact on your firm? (FY2004).............39
Research Findings - SCORE
Figure 6.1 – SCORE – Distribution of Respondents by Segment (FY2004)................................41
Figure 6.2 – SCORE – Demographic Characteristics (FY2004) .....................................................42
Figure 6.3 – SCORE – Age of Business Owner (FY2004) ..............................................................43
Figure 6.4 – SCORE – Type of Business (FY2004)..........................................................................44
Figure 6.5 – SCORE – How useful was the information you received? (FY2004) ......................45
Figure 6.6 – SCORE – Were these services useful in starting or NOT starting a business?
(FY2004) ..................................................................................................................................................46
Figure 6.7 – SCORE – As a result of assistance, have you changed management
practices/strategies? (FY2004)..............................................................................................................47
Impact Study of Entrepreneurial Development Resources
iii
Office of Entrepreneurial Development
Figure 6.8 – SCORE – Changes in Management Practices (FY2004)............................................48
Figure 6.9 – SCORE – How have these changes had an impact on your firm? (FY2004)..........49
Research Findings - WBC
Figure 7.1 – WBC – Distribution of Respondents by Segment (FY2004).....................................51
Figure 7.2 – WBC – Demographic Characteristics (FY2004)..........................................................52
Figure 7.3 – WBC – Age of Business Owner (FY2004) ...................................................................53
Figure 7.4 – WBC – Type of Business (FY2004)...............................................................................54
Figure 7.5 – WBC – How useful was the information you received? (FY2004) ...........................55
Figure 7.6 – WBC – Were these services useful in starting or NOT starting a business?
(FY2004) ..................................................................................................................................................56
Figure 7.7 – WBC – As a result of assistance, have you changed management
practices/strategies? (FY2004)..............................................................................................................57
Figure 7.8 – WBC – Changes in Management Practices (FY2004).................................................58
Figure 7.9 – WBC – How have these changes had an impact on your firm? (FY2004) ..............59
Follow-up Results
Figure 8.1 – Transition from 2003 Segments to 2004 In-Business .................................................61
Figure 8.2 – Comparison of Follow-up Results with PSED (FY2004) ..........................................63
Figure 8.3 – Revenue and Employment: 2003 vs. 2004....................................................................64
Figure 8.4 – Median Revenue by ED Resource: 2003 vs. 2004.......................................................65
Figure 8.5 – Average Employment by ED Resource: 2003 vs. 2004..............................................66
Figure 8.6 – Change in Revenues and Employment by Type of Assistance Received (FY2003)
...................................................................................................................................................................67
Appendices
Figure A.1 - Transition from 2003 Segment to In-Business: SBDC...............................................75
Figure A.2 - Transition from 2003 Segment to In-Business: SCORE............................................76
Figure A.3 - Transition from 2003 Segment to In-Business: WBC ................................................77
Impact Study of Entrepreneurial Development Resources
iv
Office of Entrepreneurial Development
TABLE OF TABLES
Table 3.1 – Sampling Frame (FY 2004) ..........................................................................................10
Table 3.2 – Follow-up Impact Study Clients (FY2003)................................................................13
Table 4.1 – Total ED Resources-Revenues, Employment, and Firm Age (FY2004)..............16
Table 4.2 – ED Resources Impact on Managerial Practices........................................................23
Table 5.1 – SBDC-Revenues, Employment, Firm Age (FY2004) ..............................................31
Table 6.1 – SCORE Revenues, Employment, and Age of Firms (FY2004) .............................41
Table 7.1 – WBC Revenues, Employment, and Age of Firms (FY2004)..................................51
Impact Study of Entrepreneurial Development Resources
v
Office of Entrepreneurial Development
I. INTRODUCTION
The Small Business Administration’s (SBA’s) Office of Business and Community Initiatives
designed a multi-year study to assess the impact of the Entrepreneurial Development (ED)
programs that it offers to small businesses. This study is part of the process of meeting the
President’s Management Agenda and subsequent evaluative processes such as the Office of
Management and Budget’s (OMB) Program Assessment Rating Tool (PART) evaluation. SBA hired
Concentrance Consulting Group to assist it in administering the study, to analyze the findings and to
write this report. This report presents the findings of the impact of ED Resources on clients served
during the fourth quarter of 2004, as well as the findings of a follow-up study of 2003 client
respondents.
This study measures attitudinal assessments, perceptions of improvement in management/marketing
skills, and business growth for firms that utilized SBA’s ED Resources during the fourth quarter of
the study year. The ED Resources include: Small Business Development Centers (SBDCs),
SCORE, and Women Business Centers (WBCs).
This report details our research as follows.
• Executive Summary
• Research questions
• Methodology
o Sampling, data collection, and survey administration
o Response rate
o Analysis of non-response
o Follow-Up
o Limitations
• Findings: 2004 Economic Impact Study
o Impact of Total ED Resources (2003 vs. 2004)
o Impact of SBDC resources 2004
o Impact of SCORE resources 2004
o Impact of WBC resources 2004
• Findings: Follow-Up Study of 2003 ED Resource Clients
• Qualitative Analysis of free-form letters, notes and comments sent in by clients
• Research Summary and Recommendations
The appendix includes the distribution of responses for each survey question, and sample survey
instruments.
Concentrance would like to thank the SBA Office of Business and Community Initiatives and SBA’s
office of Entrepreneurial Development Program Offices: Small Business Development Centers,
Women-owned Businesses, and SCORE for their assistance in conducting this research. Both
headquarter offices and local ED Resources spent considerable effort to ensure that surveys and
Impact Study of Entrepreneurial Development Resources
1
Office of Entrepreneurial Development
follow-ups were conducted. The rich dataset that now exists is in no small measure due to these
efforts.
Research Questions
The following questions were the focus of this study and are addressed in the findings section of this
report. The last two questions, #6 & #7, posed for study by the SBA are addressed in the FollowUp Study, which addresses trend analysis and requires multi-year data.
1. What is the demographic, financial, and business profile of firms who utilize SBA ED
Resources?
2. Is there a positive relationship between the services/assistance provided by SBA ED
Resources (SBDC, SCORE and WBC) and the client’s perception of the usefulness and
value of the usefulness and value of the services/assistance received?
3. Is there a positive relationship between the services/assistance provided by SBA ED
Resources (SBDC, SCORE and WBC) and the client’s decision to start a small business?
4. Is there a positive relationship between the services/assistance provided by SBA ED
Resources (SBDC, SCORE and WBC) and the client’s decision to implement and/or change
management and/or marketing practices in their business?
5. Are the services/assistance provided by SBA ED Resources (SBDC, SCORE, and WBC)
having a positive impact on the client’s financial position in terms of job creation, job
retention and sales?
Follow-Up Study of 2003 ED Resources:
6. How do the results of this survey compare with the Panel Survey of Entrepreneurial
Dynamics (PSED) database (i.e. Kauffman study)?
7. Was the financial impact (job creation, job retention and sales) of the services/assistance
provided by SBA ED Resources (SBDC, SCORE, and WBC) since last year greater than or
equal to a representative small business population?
Impact Study of Entrepreneurial Development Resources
2
Office of Entrepreneurial Development
II. EXECUTIVE SUMMARY
The Small Business Administration’s (SBAs) Office of Business and Community Initiatives designed
a study to assess the impact of its Entrepreneurial Development (ED) Resource assistance on small
business clients and hired Concentrance Consulting Group to conduct this study. This report
presents our findings of the initial impact study of 2004 clients and a follow-up study of 2003 clients.
ED Resource Clients Studied
Three ED Resource Partners were studied comprising:
• Small Business Development Centers (SBDCs),
• SCORE, and
• Women Business Centers (WBCs).
Within each ED Resource, three small business client segments were studied including:
• Nascent - Kauffman definition – those individuals who have taken one or more positive
steps to start a business;
• Start-Up - Those individuals who have been in business six months or less;
• In-Business – Those individuals who have been in business more than six months and are
classified as small by the SBA..
We compared the 2003 initial impact study results with those of 2004, analyzed the impact of ED
Resource assistance on demographic groups: gender, race/ethnicity and veteran status, as well as the
relationship of these groups to the impact of ED Resource assistance, and we assessed the financial
development and business growth of 2003 clients during 2004.
Quantitative Findings (FY2004)
Data were collected, coded, and analyzed for 3,453 respondents with an overall response rate of 23
percent. The table below provides the number of respondents and the business segment
composition of each ED Resource.
ED Resource
SBDC
SCORE
WBC
Total
Response
Stratified Usable Sample Surveys
1
Sample
Received
Rate
Size
Size
11,551
8,776
1,792
20%
7,988
5,552
1,472
27%
1,230
1,003
189
19%
20,769
15,331
3,453
23%
The stratified sample comprises all clients selected to be surveyed. The usable sample comprises stratified sample
minus surveys returned to mail house due to incorrect mailing addresses.
1
Impact Study of Entrepreneurial Development Resources
3
Office of Entrepreneurial Development
Distribution of Respondents by Segment2 (FY2004)
Number of Respondents
n=3,342
2000
1500
1000
500
0
In-Business
SBDC=52%
SCORE=43%
846 (58.4%)
1240 (72.6%)
Startup
57
Nascent
Total
WBC=5 %
410 (24.0%)
54
(3.4%)
(3.7%)
549 (37.9%)
1707
1449
Nascent
Startup
127 (68.3%)
8
(4.3%)
51 (27.4%)
186
In-Business
The 2003 clients were resurveyed to determine financial and firm growth. Their response rate was
25 percent.
ED Resource Stratified Useable Surveys Response
Center
Sample Sample Received
Rate
Size
Size3
SBDC
1,866
1,623
428
26%
SCORE
284
249
50
20%
WBC
346
288
65
23%
Total
2,496
2,160
543
25%
2 Respondents (111) who reported being in business but who did not provide a valid start date for that business, were
not assigned to a business segment. Last year these firms were classified as In-Business.
3 The stratified sample comprises all clients selected to be surveyed. The usable sample comprises stratified sample
minus surveys returned to mail house due to incorrect mailing addresses.
Impact Study of Entrepreneurial Development Resources
4
Office of Entrepreneurial Development
Clients served in 2004 were similar to clients served in 2003 with respect to
financial, business, and demographic characteristics.
¾ The financial profile of client firms by ED Resource suggests that the 2004 clients,
like the 2003 clients, represented firms of varying sizes in terms of revenues and
number of employees.
7
$250,000
6
5
$150,000
4
3
$100,000
2
Number of Employees
Annual Gross Revenue
$200,000
$50,000
1
$-
SBDC
SCORE
WBC
Revenues 2003
$196,848
$90,596
$63,694
Revenues 2004
$177,206
$115,388
0
$92,349
Employees 2003
6
2.9
2.5
Employees 2004
5.8
3.1
2.4
¾ Ed Resources had a diverse client base: females (49 percent), Black/African
American (13 percent), Veteran (10 percent), Hispanic/Latino (7 percent), Asian (4
percent), American Indian/Native Alaskan (2 percent), and Native Hawaiian/Pacific
Islander (1 percent).
¾ The highest revenues were reported by American Indian/Native Alaskan firm
owners, followed by males, whites and Hispanic/Latinos.
Impact Study of Entrepreneurial Development Resources
5
Office of Entrepreneurial Development
¾ Average revenues reported by 2004 clients increased compared to those reported by
2003 clients for all groups except for females, service disabled veterans, and Native
Hawaiian/Pacific Islanders.
¾ The primary business types for 2004 clients were the same as for 2003 clients: retail
and services.
¾ The majority of 2004 clients were between the ages of 35-54. (This demographic
characteristic was not captured for 2003 clients).
2004 respondents gave high ratings to the usefulness of the assistance they
received, however these rating were slightly lower (high value is 70 percent in
2004 versus 77 percent in 2003) than ratings in 2003.
¾ It is important to note that comparisons between 2003 and 2004 should be
interpreted with caution. Cross-sectional data were collected from a different set of
firms each year, and these firms were operating under different economic conditions.
¾ Other usefulness ratings are also slightly lower than last year: decision to start or not
to start a business, impact on changes to management practices, impact on increasing
sales, hiring new staff, and increase in profit margins.
¾ The impact on retaining current staff however, is slightly higher for 2004 clients.
Although reported impacts for clients of each ED Resource were similar for the
most part, there were some key differences.
¾ SBDC clients' perceptions of usefulness of assistance were high (74 percent reported
high value ratings), and similar to last year's ratings. However, ratings for SBDC
Nascent clients are somewhat lower than last year. SBDC clients reported higher
financial impacts than the average for all ED Resource clients combined, although
these financial impacts are somewhat smaller in 2004 than in 2003.
¾ Approximately 65 percent of SCORE respondents reported that the information
they received from their counselor was valuable, compared to 70 percent of clients
from all ED Resources combined. Interestingly, 30 percent of SCORE Start-Ups
reported that SCORE assistance resulted in their retaining current staff compared to
16 percent for all ED Resource clients. The financial impacts reported by SCORE
clients were similar to those of other ED resources, and similar to last year.
Impact Study of Entrepreneurial Development Resources
6
Office of Entrepreneurial Development
¾ Regardless of segment, WBC clients assigned higher usefulness ratings than ED
Resource clients combined. In addition, 42 percent of WBC In-Business
respondents reported that WBC assistance resulted in increased sales compared to 20
percent of ED Resource clients combined. Financial impacts were also larger for
WBC than the average for total ED Resource clients combined, but are comparable
to those reported for WBC in 2003.
Age of respondent and firm size have a significant impact on the perceived
usefulness of assistance received and on perceived assistance with the decision
to start or not to start a business.
¾ Younger clients and larger firms perceived higher usefulness of the assistance they
received.
¾ Larger firms reported higher levels of assistance with the decision to start or not to
start a business.
¾ There were significant differences in perceived financial outcomes for African
American and Veteran respondents. This impact is significantly lower than for their
counterpart groups (white and non-veteran).
ED Resource clients have a higher survival rate than those analyzed in the
PSED Kauffman study.
*
¾ The follow-up study of clients served in 2003 found that 88 percent of firms
survived from 2003 to 2004, whereas the survival rate of firms in the PSED sample
was 82 percent.
ED Resource clients had revenue growth of approximately 9 percent and an
average increase in number of employees by 19 percent.
*
¾ Revenue growth for these firms compares favorably with the growth in real GDP of
3.8 percent during this period.
¾ Clients from all three ED Resources saw significant firm development from 2003 to
2004.
• 24 percent of Nascent SBDC clients were In-Business in 2004
• 27 percent of Nascent SCORE clients were In-Business in 2004
• 100 percent of Nascent WBC clients were In-Business in 2004
* According to the Authors, the follow-up sample meets some ad hoc but not formal statistical criteria. Further study
is warranted to support the conclusions.
Impact Study of Entrepreneurial Development Resources
7
Office of Entrepreneurial Development
Qualitative Findings
¾ A few clients from each ED Resource provided ad hoc commentaries in letter form
or in the form of hand written note on the survey instrument.
¾ Comments were a mix of positive and negative for each ED Resource with no
particular outstanding issue stressed.
Recommendations
Our recommendations focus primarily on improving response rates to the follow up survey and
include the following:
¾ Using telephone versus mail for the follow-up surveys,
¾ Shortening the survey by eliminating questions that have been asked previously and
do not comprise trend data.
Methodology
The initial study of 2004 clients was quantitative in nature, based upon responses to mail and
telephone surveys at one point in time. Statistical analyses utilized frequency distributions, mean
comparisons, cross tabulations, and regression models.
The initial survey of 2004 clients resulted in a respondent sample of 3,453. ED Resources provided
Concentrance with a random sample list (proportionately distributed across ED Resources) of
clients served during the fourth quarter of 2004. Surveys were mailed to each client and mailed
again to non-respondents ten days later. Phone surveys were attempted for clients who did not
respond to mail surveys. According to statistical tests, there was no evidence of bias due to nonresponse, therefore mail and phone surveys were analyzed together.
Regression analyses were used to test for the impact of client demographics on the outcomes of the
ED Resources.
The respondent sample for the follow-up survey of 2003 clients was 5434 with 2055 having valid
client codes enabling us to match them to the previous survey. We used the matched sample to
perform pair-wise statistical tests that compare changes in growth and business segment to other
groupings.
This is the number for which there was valid client telephone or address information provided by ED Resource
Centers.
5 This is the number of clients for which there were valid client codes to match.
4
Impact Study of Entrepreneurial Development Resources
8
Office of Entrepreneurial Development
Limitations to consider when interpreting these results include:
¾ Measures of ED Resource impacts are attitudinal rather than objective, however
objective data on financial outcomes are harder to collect;
¾ Many respondents did not report revenue data;
¾ Outcomes do not include consideration of the amount of assistance received since
ED Resource utilization is not measured, and
¾ Financial and growth outcomes for the follow-up group should be viewed with
caution due to the small sample size.
Impact Study of Entrepreneurial Development Resources
9
Office of Entrepreneurial Development
III. METHODOLOGY
This report contains findings from the 2004 Impact Study of ED Resources and the Follow-Up
Study of the 2003 Initial Impact Survey. The methodology for the 2004 Impact Study is presented
first, followed by the methodology for the 2003 Follow-Up Study.
Sampling and Data Collection
We examined the research questions by SBA ED Resource and by clients in various stages of
business formation:
• Nascent - Individuals who have taken one or more steps to start a business
• Start-Up - Individuals who have been in business six months or less
• In-Business – Individuals who have been in business more than six months and are classified
as small by the SBA
The sampling distribution is shown in Table 3.1. It represents the distribution of clients served in
2004 across ED Resources. This sample provides a statistically significant number of respondents
by ED Resource. To determine if the sample size for this study was sufficient, sample size was
computed on the basis of reported sales revenues. The sample size of 3,453 is sufficiently large
enough for accuracy within plus or minus $30,000 in reported annual sales revenues at a 99 percent
confidence level.
Table 3.1 below depicts the distribution of surveys across ED Resource and the response rates by
ED Resource. The overall response rate was 23 percent, which is consistent with response rates
found in other surveys of entrepreneurs. For example, Aldrich and Baker (2000) found that nearly a
third of surveys of entrepreneurs had response rates below 25 percent, and that the average response
rate is approximately 30 percent.6 In another example, the National Federation of Independent
Business (NFIB) has a response rate of 25 percent in its Survey of Small Business Economic Trends
(Dennis 2003).7
Table 3.1 – Sampling Frame (FY 2004)
ED Resource
Center
SBDC
SCORE
WBC
Total
Stratified
Useable
Surveys
Response
Response
Rate
Rate
Sample Sample Size8 Received
Size
(FY2004)
(FY2003)
11,551
8,776
1,792
20%
23%
7,988
5,552
1,472
27%
22%
1,230
1,003
189
19%
46%
20,769
15,331
3,453
23%
24%
Aldrich, Howard E. and Ted Baker (2000), “Blinded by the Cites: Has there been any progress in entrepreneurship
research?” in Entrepreneurship 2000, ed. Donald L. Sexton and Raymond L. Smilor, Chicago: Upstart Publishing, p. 377
400.
7 Dennis, William J., Jr. (2003), “Raising response rates in mail surveys of small business owners: Results of an
Experiment,” Journal of Small Business Management, July, 41(3), pg. 278
8 Stratified sample comprises all clients selected to be surveyed. Usable sample comprises stratified sample minus surveys
returned to mail house due to incorrect mailing addresses.
6
Impact Study of Entrepreneurial Development Resources
10
Office of Entrepreneurial Development
Survey Administration
Concentrance developed a random sample table for each ED Resource. SBA ED Resource
personnel, using the random sample table, selected clients served within the fourth quarter of 2004.
See Appendix 4 for survey instrument. Concentrance printed, and mailed SBA’s cover letters and
survey instruments, approved by OMB. All surveys were computer-coded with client codes to
ensure that a second mailing was sent to non-respondents ten days after the initial mailing and that
information would remain confidential. Concentrance logged-in all received surveys, entered the
surveys into a database for analysis and developed an electronic database list of non-respondents for
telephone follow-up.
Figure 3.1 shows the distribution of respondents by resource and business segment. The In-Business
segment was the largest segment served for all resources. The Start-up segment is only 4 percent, in
part because of the narrow definition of Start-ups (firms that have been in business for 6 months or
less) and in part because a valid start date for the business must be available to identify this segment.
Figure 3.1 – Survey Sample Sizes by ED Resource and Segment9
(FY2004)
Number of Respondents
n=3,342
2000
1500
1000
500
0
In-Business
SBDC=52%
SCORE=43%
846 (58.4%)
1240 (72.6%)
Startup
57
Nascent
Total
WBC=5 %
54
410 (24.0%)
(3.4%)
(3.7%)
549 (37.9%)
1707
1449
Nascent
Startup
127 (68.3%)
8
(4.3%)
51 (27.4%)
186
In-Business
Respondents (111) who reported being in business but did not provide a valid start date for that business were not
assigned to a business segment. Last year these firms were classified as In-Business.
9
Impact Study of Entrepreneurial Development Resources
11
Office of Entrepreneurial Development
Data Analysis
Analyses included the following methods: frequency distributions, mean comparisons, cross
tabulations and regressions. Frequency distributions were used for Likert scales where respondents
answered on a scale of 1 to 5. Mean comparisons were conducted to analyze continuous variables,
e.g. annual sales revenues. Cross-tabulations were used to compare responses across resources and
segments. Finally, multiple regression analyses were conducted to test for differences in key
outcomes, such as perceived usefulness or financial impact of ED Resource assistance, based on
demographic or firm characteristics. These analyses were selected in order to reflect the objectives
of each research question and to facilitate the interpretation of the findings.
Analysis of Non-Response
SBA’s research plan for this study included a methodology for estimating potential non-response.
Non-response bias can occur in survey research if respondents differ substantially from those who
do not respond. The best way to protect against this bias is to improve the response rate. Another
effective method for achieving high response rates is the use of follow-ups and reminders. In this
study, non-response bias was minimized by the second mailing of the survey and further by
telephone follow-ups to non-respondents.10
In addition, we assessed the extent and direction of potential non-response bias by studying the
differences between those who responded and those who did not. Our approach was to conduct a
phone survey of non-respondents. A total of 544 phone surveys were conducted with mail nonrespondents. The next step was to conduct a comparative analysis of the responses of the
two groups.
Although there were some differences between these groups, there is no strong evidence to
conclude that there is significant bias due to non-response. Results of these analyses indicated that
the differences between the mail and phone samples were statistically significant in some cases, but
not significant in others.11 For example, mail and phone respondents differed significantly in terms
of annual revenues and number of employees. Phone respondents were significantly less likely than
mail to report annual revenues. In addition, phone respondents on average have fewer employees
than mail respondents. On the other hand, results of chi-square contingency tests revealed no
significant association between mode of data collection (mail vs. phone) and business segment.
However, because these differences were small and did not pose any risk of systematic bias
in the results, responses from the mail and phone survey administrations were combined in
all analyses.
Research in this area suggests that the optimum number of follow-up mailings or reminders is two.
These findings were based on results of a one-way analysis-of-variance (ANOVA) to determine whether mail versus
phone respondents differed significantly in terms of sales revenues (t=10.96, dfs=1; p=0.0001). This was a test of the
null hypothesis that the average sales revenues were the same between these groups. A p-value less than 0.01 suggests
that there is evidence of significant differences between mail and phone respondents. A second ANOVA was
conducted to determine whether mail versus phone respondents differed significantly in terms of number of employees
(t=02.039, dfs=1;,p=0.0415). This was a test of the null hypothesis that the average number of employees was the same
between these groups. A p-value greater than 0.01 suggests that there is no statistically significant difference between
mail and phone respondents.
10
11
Impact Study of Entrepreneurial Development Resources
12
Office of Entrepreneurial Development
Follow-Up Study – Sampling, Data Collection, and Analysis of Non-Response
There were 4,718 respondents to the 2004 survey. A follow-up survey of these respondents was
conducted in 2005 by phone. The follow-up survey sample included 543 respondents. While this
represents only 11 percent of the overall Initial Impact Study sample, it represents a 25 percent
response rate based on respondents for whom useable address or telephone data was available.
Table 3.2 shows response rates by ED Resource.
Table 3.2 – Follow-up Impact Study Clients (FY2003)
ED Resource Stratified Useable Surveys Response
Center
Sample Sample Received
Rate
Size
Size12
SBDC
1,866
1,623
428
26%
SCORE
284
249
50
20%
WBC
346
288
65
23%
Total
2,496
2,160
543
25%
Out of the 543 completed follow-up surveys, 205 had valid client codes that could be matched on a
one-to-one basis with the initial impact survey responses. Thus, for a sample of 205 clients, there
were detailed data which could be tracked from 2003 to 2004.
The matched sample was used to perform pair-wise statistical tests that compared changes in growth
by segment, ED Resource, and other groupings. Although there were fewer observations available
for this analysis, findings provided a more precise estimate of changes in revenues and employment
over time than aggregate, non-matched comparisons.
Non-response bias can occur in follow-up surveys if the characteristics of the initial respondent
sample differ significantly from the follow-up sample. The extent and direction of potential nonresponse bias were assessed by comparing the two samples. A statistical comparison of average
revenues reveals that there is no statistically significant difference between these samples.13
The stratified sample comprises all clients selected to be surveyed. The usable sample comprises stratified sample
minus surveys returned to mail house due to incorrect mailing addresses.
13 t=1.127, df=3,658, p=0.2596.
12
Impact Study of Entrepreneurial Development Resources
13
Office of Entrepreneurial Development
Limitations (FY2003 Follow-up)
There are several limitations that should be considered when interpreting the results presented in
this report. These limitations are primarily posed by the research design and methodology.
¾ Measures included in this survey are primarily attitudinal and all items are selfreported. Thus, there are few objective measures that can be used to assess actual client
performance or make comparisons with other data sources.
¾ Many respondents did not report revenue data, which is critical to measure growth from
year-to-year. For example, out of Start-Up and In-Business firms, approximately 63 percent
reported zero revenues or did not report revenues.
¾ This survey does not measure ED Resource utilization. Thus, we cannot measure the
relationship between the amount of assistance provided by ED Resources and key
outcomes, such as financial impacts. In other words, these analyses cannot differentiate
between clients who had limited contact with a counselor and clients who had more
extensive assistance. This problem would be alleviated if the agency provided, for example,
data on number of hours of assistance. The ability to conduct regression analysis, causal
modeling, and other statistical measures of relationships are similarly limited.
¾ The follow-up study results must be interpreted with caution due to the small size of
the sample that could be matched with last year’s returned surveys. The size of this
sample was limited because of missing or incorrect/inconsistent client codes needed to
match each respondent one-for-one with last year’s survey and because numerous surveys
were returned from the mail house due to incorrect addresses. This problem has been
alleviated for subsequent surveys since all respondents are assigned a permanent client code
by Concentrance at the time the first survey of the respondent is received. In addition, for
future mailings, the post office will correct address changes for the prior 36 months prior to
mailing surveys. Conducting all follow-ups using an abbreviated follow-up survey via phone
should increase response rates. This approach is also faster and more efficient than
conducting mail follow-ups (Dennis 2003).
¾ Finally, since this was the first follow-up to the initial or baseline survey, respondents may
have had insufficient time in which to realize the effects of ED Resource assistance.
.
Impact Study of Entrepreneurial Development Resources
14
Office of Entrepreneurial Development
IV. TOTAL ED RESOURCES COMBINED 2003 VS. 2004
The following section presents comparison results of clients from 2003 versus 2004. We have
included 2003 clients from Pennsylvania SBDC not included in last year’s report and we have
excluded 503 BIC clients from 2003 totals. First, demographic characteristics of the sample are
presented, followed by additional findings organized by research question, as follows:
• Is there a positive relationship between the services/assistance provided by ED Resources
and the client’s perception of the usefulness and value of the services/assistance received?
• Is there a positive relationship between the services/assistance provided by ED Resources
and the client’s decision to start a business?
• Is there a positive relationship between the services/assistance provided by ED Resources
and the client’s decision to implement and/or change management and/or marketing
practices in their business?
• Is there a positive relationship between the services/assistance provided by ED Resources
and the client’s business having a positive financial impact on job creation and retention and
increase in sales?
It is important to note that comparisons between 2003 and 2004 clients should be interpreted with
caution. Cross-sectional data were collected from a different set of firms each year, and these firms
were operating under difference economic conditions. Cross-sectional research has limited
usefulness for examining changes over time since the data does not provide information about
sequential changes in variables.
Demographic Characteristics of ED Resource Clients
This section describes the general demographic and financial characteristics of all ED Resource
respondents who were served in 2004. These results are based on an analysis of 3,453 completed
surveys.14
14
The sample size in 2004 was 4,718 and included data from BICs. These data were not collected in 2005.
Impact Study of Entrepreneurial Development Resources
15
Office of Entrepreneurial Development
As shown in Figure 4.1, the largest business segment served by ED Resources was the In-Business
segment (66 percent). In 2003, 62 percent of ED Resource clients were in this segment. This
difference is due to the fact that ED Resources served fewer Nascent clients in 2004 (30 percent vs.
34 percent in 2003). The percentage of Start-Ups was similar in both years.
Figure 4.1 – Total ED Resources – Client Segments Served
70%
66%
62%
60%
50%
40%
33%
30%
30%
20%
10%
5%
4%
0%
Nascent
Start-up
2003
In-Business
2004
ED Resource clients averaged $148,952 in annual revenues. In 2004, this figure was $134,53915.
Average total employees did not change from last year. On average, In-Business clients have been
operating for just over 6 ½ years.
Table 4.1 – Total ED Resources-Revenues, Employment, and Firm Age16
(FY2004)
Average Revenue
Average Employees
Firm Age (in years)
InTotal ED
Startup Business Resources
$44,987 $160,473 $148,952
1.2
5.0
4.6
0.4
6.7
6.4
15 This estimate differs from average revenues reported in last year’s report. This revised estimate includes data from
Pennsylvania clients, and excludes data from BICs. In addition, 7 firms who reported revenues over $8 million were
considered outliers and were excluded from this estimate.
16 Firms who reported revenues of $8 million or more were considered outliers and were omitted from revenue analyses.
There were 6 firms with revenues over $8 million in 2004.
Impact Study of Entrepreneurial Development Resources
16
Office of Entrepreneurial Development
ED Resources differed in terms of the size of firms they serve, as shown in Figure 4.2. For
example, SBDC clients were significantly larger than those served by SCORE and WBC, in terms of
revenues and employees. SBDC clients have somewhat lower revenues and fewer employees than
last year. SCORE and WBC clients have slightly higher revenues and similar number of employees
as last year.
Figure 4.2 – ED Resources – Revenue and Employment
$250,000
7
6
5
$150,000
4
3
$100,000
2
Number of Employees
Annual Gross Revenue
$200,000
$50,000
1
$-
SBDC
SCORE
WBC
Revenues 2003
$196,848
$90,596
$63,694
Revenues 2004
$177,206
$115,388
0
$92,349
Employees 2003
6
2.9
2.5
Employees 2004
5.8
3.1
2.4
Impact Study of Entrepreneurial Development Resources
17
Office of Entrepreneurial Development
Demographic characteristics of ED Resource clients in terms of gender, race, ethnicity, and veteran
status, are presented in Figure 4.3. These demographic characteristics differed only slightly from
2004 clients. Females represented 49 percent of this year’s sample compared to 46 percent last year.
This year, 13 percent of respondents were African-American and 7 percent were Hispanic or Latino,
compared to 13 and 6 percent respectively last year. The share of veterans and service-disabled
veterans was also similar to last year’s.
Figure 4.3 – Total ED Resources – Demographic Characteristics
6%
7%
Hispanic/Latino
63%
Race/Ethnicity
White
Native Hawaiian/Pacific Islander
69%
1%
1%
13%
13%
Black/African American
4%
4%
Asian
3%
2%
Veteran
Gender Status
American Indian/Native Alaskan
Service Disabled Veteran
1%
1%
11%
10%
Veteran
46%
49%
Female
48%
47%
Male
0%
2003
10%
20%
30%
40%
50%
60%
70%
80%
2004
Impact Study of Entrepreneurial Development Resources
18
Office of Entrepreneurial Development
Figure 4.4 shows the distribution of ED Resource clients by age of the business owner. The largest
group of ED Resource clients was the 45 to 54 category (30.6 percent). These data were not
available in last year’s survey.
Figure 4.4 – Total ED Resources – Age of Business Owner (FY2004)
75 +
1%
Respondent Age (Years)
65-74
3%
55-64
15%
45-54
31%
35-44
26%
25-34
18-24
0%
16%
2%
5%
10%
15%
20%
25%
30%
35%
Impact Study of Entrepreneurial Development Resources
19
Office of Entrepreneurial Development
The two most common types of businesses served by ED Resources are service and retail
businesses. These results are similar to last year, although 2004 respondents could choose from
several additional business categories that were not available previously. These estimates are
consistent with the top 2 employment sectors reported by the Bureau of Labor Statistics17 (BLS),
although the industry classifications are not identical. According to the BLS, the most common
employment sectors in 2004 were retail and wholesale trade establishments (16 percent) and
professional and business services (12 percent).
Figure 4.5 – Total ED Resources – Primary Business Type (FY2004)
15%
Service
10%
Retail
Restaurant
3%
9%
Others
5%
Manufacturing
3%
Healthcare
Finance/Insurance/Real Estate
2%
Education
2%
Consulting
4%
4%
Construction
0%
17
4%
8%
12%
16%
Source: “Industry Employment,” Occupational Outlook Quarterly, U.S. Bureau of Labor Statistics, Winter 2005-06.
Impact Study of Entrepreneurial Development Resources
20
Office of Entrepreneurial Development
Resource Impacts
Impact on Usefulness and Value of Services or Assistance Received
As shown in Figure 4.6, 70 percent of ED Resource respondents combined reported that the
information they received from their counselor was valuable. However, perceived usefulness ratings
were slightly lower than those reported last year.
Figure 4.6 – Total ED Resources – How useful was the information
you received?18
90%
77%
80%
70%
70%
60%
50%
40%
30%
20%
18%
13%
10%
4%
4%
0%
Low
Medium
2003
High
2004
Low value includes the ‘Somewhat Valuable’ and ‘Not Valuable’ response categories. High value includes the
‘Extremely Valuable’ and ‘Valuable’ response categories.
18
Impact Study of Entrepreneurial Development Resources
21
Office of Entrepreneurial Development
Impact on the Decision to Start or Not Start a Business
ED Resources assisted clients in their decision to start a business for 34 percent of respondents.
Last year, 43 percent of respondents reported that ED Resources assisted them in their decision to
start a business. In addition, 12 percent of this year’s respondents reported that ED Resource
assistance helped them decide not to start a business, and this is about the same as last year. In cases
when individual entrepreneurs lack resources such as knowledge or expertise necessary to create a
successful start-up, the prevention of business failures is an important type of ED resource
assistance. In particular, these activities are part of the SBA’s strategic plan that calls for increasing
the number and success of small business start-ups, and maximizing the sustainability of existing
small businesses19.
Figure 4.7 – Total ED Resources – Were these services useful in
starting or NOT starting a business?20
50%
43%
40%
34%
30%
20%
15%
12%
10%
0%
Helped me decide to start a business
Helped me decide NOT to start a business
2003
2004
“Enabling the Establishment and Viability of Small Businesses: Strategic Plan FY2003-FY2008,” U.S. Small Business
Administration.
20 Helpful includes the ‘Very Useful’ and ‘Useful’ response categories only.
19
Impact Study of Entrepreneurial Development Resources
22
Office of Entrepreneurial Development
Impact on Clients’ Decision to Change Management Practices
As a result of ED Resource assistance, 51 percent of Start-Up and 58 percent of In-Business firms
have changed their current management practices or strategies. These figures are somewhat lower
than in the previous year, when 63 percent of Start-ups and 64 percent of In-Business firms changed
their management practices.
Table 4.2 – ED Resources Impact on Managerial Practices
Total ED Resources
Start Up
In-Business
2003
Changed current management
practices/strategies
64%
63%
64%
2004
Changed current management
practices/strategies
57%
51%
58%
Figure 4.8 shows the business functions that were changed as a result of ED Resource assistance.
Business plans and marketing plans were the most frequently cited functions, followed by cash flow
analysis and financial strategies. These areas were similar to those reported last year.
Figure 4.8 – Total ED Resources – Changes in Management Practices
35%
32%
30%
26%
25%
22%
20%
18%
15%
15%
11%
10%
11%
8%
10%
8%
6%
4%
5%
9%
11%
9%
12%
5%
4%
3%
3%
Cash Flow
E-Commerce
2004
Financial
management
Promotional
plan
2003
Pricing
Feasibility
Hiring
Marketing
plan
Loan
package
Business
Plan
0%
Impact Study of Entrepreneurial Development Resources
23
Office of Entrepreneurial Development
Impact on Sales, Job Creation and Job Retention
Figure 4.9 presents findings on the economic impact of ED Resource assistance. For example, 20
percent of respondents reported that they were able to increase sales as a result of ED Resource
assistance, and 9 percent were able to hire new staff. The perceived impact of ED Resource
assistance on sales, hiring new staff and profits is somewhat smaller than for 2003 clients, but the
perceived impact on staff retention is higher for 2004 clients.
Figure 4.9 – Total ED Resources – How have these changes had an
impact on your firm?
30%
27%
25%
25%
23%
19%
20%
15%
11%
12%
13%
14%
10%
5%
0%
Increase sales
Hire new staff
2003
Retain current staff
Increase profit
margin
2004
Impact Study of Entrepreneurial Development Resources
24
Office of Entrepreneurial Development
Cross-Sectional Demographic Analysis
This section compares revenues and employment for ED Resource clients by gender, race, ethnicity,
and veteran status.
Figure 4.10 compares average revenues for firms in each demographic category. For example, this
figure shows that women-owned and minority-owned firms (except for American Indian/Alaskan)
tended to be smaller than firms owned by males or Whites. The average women-owned business
has $83,857 in revenues compared to an average of $223,179 for firms owned by men. In terms of
minority-owned firms, average revenues for Hispanic/Latino-owned firms were $160,739; $149,250
for Asian-owned firms; and $42,250 for Black/African-American-owned firms. In 2004, firms
owned by American Indian/Alaskans had the largest revenues, although the size of this sub-sample
is very small (n=50). These figures should be interpreted with caution due to the fact that almost
30 percent of respondents did not report revenue data. There was a similar incidence of data not
reported on race and ethnicity.
Figure 4.10 – Average Revenues (000s) for Firms in Selected
Demographic Categories21
$350
$286
$ Revenue (000s)
$300
$250
$223
$206
$200 $188
$137
$150
$87
$100
$149
$122
$110
$85
$84
$171
$161
$152
$140
$119
$44 $42
$50
$7 $1
hi
te
W
r
n/
Pa
c
ifi
c
Is
la
er
Am
ac
k
Bl
nd
e
ic
an
ia
n
/A
fri
ca
n
ia
n
In
d
er
ic
an
Am
2003
H
aw
ai
ia
/L
a
ni
c
As
tin
/A
ls
ka
n
o
le
d
ab
ce
-D
is
H
is
pa
Se
rv
i
Ve
te
ra
n
m
al
es
Fe
M
al
e
s
$0
2004
Firms who reported revenues of $8 million were considered outliers and were omitted from revenue analyses from
both 2003 and 2004. There were 6 firms with revenues over $8 million in 2003 and 7 in 2004.
21
Impact Study of Entrepreneurial Development Resources
25
Office of Entrepreneurial Development
Figure 4.11 compares employment by demographic category. Similar to revenues, women-owned
businesses tended to have fewer employees than firms owned by males. Male, Hispanic-owned,
white and Veteran-owned firms had more employees than the overall average (4.6 percent).
Figure 4.11 – Average Total Employees for Firms in Selected
Demographic Categories
7
6.0
# of Employees
6
5.6
5.4
5.3
4.8
5
3.7
4
4.8 4.9
4.8
4.0
3.8
3.5
2.9
2.7
3
2.8
2.6
2.3
1.8
2
1.6
1.4
1
2003
hi
te
W
an
de
r
Is
l
H
aw
ai
ia
n/
Pa
ci
fic
er
ic
an
n
an
-A
m
As
ia
ac
k/
Af
ric
Bl
Se
rv
ic
eD
is
ab
le
d
H
is
pa
Am
ni
c/
er
La
ic
tin
an
o
In
di
an
/A
la
sk
an
Ve
te
ra
n
al
es
Fe
m
M
al
es
0
2004
Regression Analysis: Effects of Gender, Race, Ethnicity, and Veteran Status on Key ED
Resource Impacts
This section presents findings of an analysis of the effects of firm demographic characteristics on
key resource impacts. We report results from a series of multiple regression analyses. These were
conducted to test whether respondents differed significantly in perceived usefulness, perceived
assistance with the decision to start a business, and financial impacts of ED Resource assistance
based on selected demographic (age, gender, race, ethnicity, veteran status) and firm (revenues,
business segment, ED Resource) characteristics. This approach is different from last year, when
different models were estimated separately for each ED Resource. This year, the data was combined
Impact Study of Entrepreneurial Development Resources
26
Office of Entrepreneurial Development
across all resources in order to increase the amount of data available from smaller demographic
categories.
These analyses address the following research questions:
1. Are there differences in perceived usefulness based on age, gender, race, ethnicity, or veteran
status?
2. Are there differences in perceived usefulness based on firm size or business segment?
3. Are there differences in perceived assistance with the decision to start a business based on age,
gender, race, ethnicity, or veteran status?
4. Are there differences in perceived assistance with the decision to start a business based on firm
size or business segment?
5. Are there differences in the financial impact of ED Resource assistance based on age, gender,
race, ethnicity, or veteran status?
6. Are there differences in the financial impact of ED Resource assistance based on firm size or
business segment?
Each regression model examined the impact of selected demographic characteristics (age, gender,
race, ethnicity and veteran status) and firm characteristics (size, business segment, and ED Resource)
on a particular resource impact. These impacts included: perceived usefulness of ED Resource
assistance, the decision to start a business, and perceived impact on sales.
Each resource impact is used as a dependent variable. For example, the dependent variable for the
first model is perceived usefulness of ED Resource assistance. The dependent variable for the
second model is perceived usefulness in the decision to start a business. The dependent variable for
the third model is a financial impact score, which is a summary measure based on 4 self-reported
financial impact measures (the effect of assistance on increases in market share, sales, cash flow, and
profit margins).22
The set of independent or predictor variables included binary (i.e. dummy) variables representing the
demographic or firm characteristics including:
• Respondent age
• Gender
• Race
o Hispanic
o American Indian
o Asian
o Black/African-American
This scale is the sum of the following questionnaire items: q7a, q7b, q7c and q7g. This scale has a Cronbach’s alpha
coefficient of 0.9513, which reflects high internal consistency of items, and is considered an indicator of very good scale
reliability.
22
Impact Study of Entrepreneurial Development Resources
27
Office of Entrepreneurial Development
•
•
•
•
•
o Native Hawaiian/Pacific Islander
Ethnicity
o Hispanic/Latino
Veteran or Service-Disabled Veteran
Client Segment
o Start-Up
o In-Business
Sales Revenues
ED Resource (SBDC, SCORE, or WBC)
The first regression model tested for a relationship between selected demographics and firm
characteristics and the client’s perception of the usefulness and value of the services/assistance
received.23
Results of this regression suggest that there is a significant difference in perceived usefulness of ED
Resource assistance based on age of respondent. In particular, there is a significant and
negative relationship between client age and perceived usefulness. In other words, older
clients report lower levels of perceived usefulness. This may be due to the fact that older
respondents had more business acumen than younger respondents. There were no significant
effects of gender, race, ethnicity, or veteran status on perceived usefulness.
There is a significant and positive relationship between firm size and perceived usefulness. Larger
firms reported higher levels of perceived usefulness. There were no significant effects of
business segment on this outcome. Finally, perceived usefulness was significantly lower for SCORE
clients than those of SBDC or WBC regardless of firm characteristics.
The second regression model tested for a relationship between selected demographics (age, gender,
race, ethnicity, veteran status) and firm characteristics (revenues, business segment, and ED
Resource) and the perceived usefulness of assistance with the client’s decision to start a business.24
Results of this regression suggest that there is a significant difference in perceived usefulness of
assistance with the decision to start a business based on age of respondent. In particular, there is
a significant and negative relationship between client age and assistance with starting a
business. In other words, older clients report lower levels of usefulness of assistance with starting a
business. There were no significant effects of gender, race, ethnicity, or veteran status on
perceived usefulness of assistance with starting a business.
There is also a significant and positive relationship between firm size and perceived usefulness of
assistance with starting a business. In particular, larger firms reported higher levels of
perceived usefulness of assistance with decision to start a business. There were no significant
effects of business segment on this outcome. Finally, perceived usefulness of assistance with
The overall model was significant at the p
